A systems-based coherence framework for interpreting expert knowledge in ex ante adoption assessment

A systems-based coherence framework for interpreting expert knowledge in ex ante adoption assessment | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A systems-based coherence framework for interpreting expert knowledge in ex ante adoption assessment Liliam Marquínez-Batista, Jaime Espinosa-Tasón, Mariana Cruz-Chú, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9383379/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Accurate interpretation of expert-based assessments is critical for decision-making in complex socio-technical systems, particularly in ex ante contexts where empirical data are unavailable. In such settings, expert knowledge is often expressed through multiple representations that differ in structure, transparency, and level of aggregation. However, the relationships among these alternative representations remain poorly understood, limiting their effective use in decision support. This study proposes a systems-based coherence framework to interpret structured adoption modeling and holistic expert judgment as alternative encodings of a shared latent adoption process. Fourteen senior experts from Panama’s agricultural research and extension system evaluated established production systems using both the Adoption and Diffusion Outcome Prediction Tool (ADOPT) and direct judgment-based assessments. Multivariate alignment between representations was evaluated using Canonical Correlation Analysis as a diagnostic tool, complemented by leave-one-expert-out influence analysis and clustered bootstrap resampling interpreted as stability diagnostics. Results show weak and inconsistent correspondence at the level of individual parameters, but a moderate, directionally stable multivariate association (ρ = 0.47) between the representations. Robustness analyses indicate that this coherence persists across alternative expert configurations, despite heterogeneity in individual influence. These findings suggest that agreement between expert-based assessments should be evaluated as a system-level property rather than through parameter equivalence. The proposed framework provides a practical basis for interpreting expert-based assessments in decision-support contexts characterized by uncertainty and limited data. technology adoption expert elicitation ADOPT canonical correlation analysis ex ante assessment decision support Figures Figure 1 Figure 2 Figure 3 1. Introduction Ex ante assessment of agricultural technology adoption plays a central role in research prioritization, extension planning, and policy design. In complex socio-technical systems, such assessments rely heavily on expert knowledge to anticipate system behavior under uncertainty, particularly when empirical data are unavailable or incomplete. Because adoption processes unfold over long time horizons, decision-making frequently depends on how expert knowledge is elicited, structured, and interpreted across different analytical approaches (Feder et al. 1985 ; Prokopy et al. 2019 ; Burke et al. 2020 ). A substantial body of empirical research has examined the determinants of agricultural technology adoption using farm-level data and econometric methods. These studies show that adoption is shaped by interacting economic, biophysical, social, and institutional factors, and that estimated effects are highly context-dependent (Doss and Morris 2001 ; Sunding and Zilberman 2001 ). More recent syntheses further highlight that drivers of adoption are sensitive to how adoption processes are conceptualized and measured, complicating comparisons across studies and analytical tools (Prokopy et al. 2019 ; Ruzzante et al. 2021 ). In ex ante contexts, where longitudinal adoption data are unavailable, expert elicitation has become an essential source of information for anticipating adoption dynamics (Aspinall 2013 ; Hanea et al. 2017 ). However, elicitation paradigms differ in how they structure expert knowledge, raising questions about how their outputs should be interpreted when applied to the same problem. Structured approaches formalize knowledge through predefined determinants embedded in analytical models, whereas holistic approaches elicit direct judgements that integrate contextual and experiential knowledge without imposing explicit structure. The Adoption and Diffusion Outcome Prediction Tool (ADOPT) is a widely used structured approach that translates expert assessments of adoption determinants into parametric diffusion trajectories (Kuehne et al. 2017 ; Llewellyn et al. 2018 ). In contrast, holistic expert judgement provides direct estimates of adoption outcomes without explicit decomposition of underlying drivers. These approaches encode expert knowledge differently, making direct comparison at the level of individual parameters conceptually problematic. Structured models distribute information across multiple determinants, whereas holistic judgements compress interacting factors into a smaller set of outcome estimates. Existing comparisons between these paradigms often assume that agreement should emerge at the level of individual parameters, such as adoption ceilings or timing. This assumption may be misleading because it ignores structural differences in how adoption dynamics are represented. Apparent discrepancies at the parameter level may therefore reflect differences in representation rather than disagreement about the underlying process. This study addresses this issue by proposing a systems-based coherence framework for interpreting expert-based adoption assessments in ex ante contexts. Rather than testing for parameter equivalence, we evaluate whether structured adoption modeling and holistic expert judgement exhibit a stable multivariate association that reflects a shared latent conception of adoption dynamics. Canonical Correlation Analysis (CCA) is used as a diagnostic tool to assess alignment between representations, complemented by leave-one-expert-out (LOEO) influence analysis and clustered bootstrap resampling interpreted as stability diagnostics (Hardoon et al. 2004 ). The empirical analysis is based on a census of 14 senior experts from Panama’s agricultural research and extension system who evaluated established production systems using both elicitation paradigms. Because the expert panel constitutes a bounded population rather than a probabilistic sample, robustness is assessed through diagnostics of stability and influence rather than inferential statistics (Aspinall 2013 ; Hanea et al. 2017 ). By reframing agreement between elicitation paradigms as a question of multivariate coherence and stability, this study provides a practical basis for interpreting expert-based adoption assessments in decision-support contexts characterized by uncertainty and limited data. 2. Conceptual framework Ex ante assessment of agricultural technology adoption requires translating expert knowledge into quantitative representations to inform research prioritization and policy decisions. In systems terms, these representations can be understood as alternative ways of encoding knowledge about the behavior of a complex socio-technical system. Because these representations differ in structure and level of aggregation, they may not be directly comparable at the level of individual parameters. The framework developed here distinguishes between structured adoption modeling and holistic expert judgement and defines their coherence as a system-level property. 2.1. Alternative encodings of expert knowledge Structured adoption models decompose adoption dynamics into predefined determinants that jointly shape diffusion processes. From a systems perspective, this approach makes explicit the internal structure of the representation by separating interacting components. Tools such as ADOPT operationalize this paradigm by eliciting standardized assessments of dimensions such as relative advantage, learning requirements, population heterogeneity, and contextual constraints, which are then mapped onto a parametric adoption curve. Adoption outcomes emerge from the interaction among these determinants within a coherent model structure. In contrast, holistic expert judgement elicits direct assessments of adoption outcomes—such as expected adoption ceilings or timing—without imposing an explicit internal structure. Experts integrate contextual knowledge, experience, and tacit understanding, but the weighting of underlying determinants remains implicit. This results in a compressed representation in which interactions are embedded but not explicitly parameterized. Both paradigms aim to represent the same underlying phenomenon: the expected trajectory of technology adoption within a given system. Rather than constituting independent sources of information, they can be interpreted as alternative encodings of a shared latent conception of adoption dynamics. Differences between their outputs, therefore, reflect how knowledge is structured rather than necessarily indicating disagreement. 2.2. Coherence versus parameter correspondence Evaluations of expert-based adoption assessments often assume that coherence should be reflected in correspondence between individual parameters, such as similar estimates of adoption ceilings or time to widespread uptake. This assumption is problematic because it ignores structural differences between representations. Adoption is inherently multivariate, shaped by interacting determinants that jointly influence diffusion trajectories. Structured models distribute this information across multiple parameters, whereas holistic judgements integrate these interactions into fewer outcome estimates. As a result, parameters across paradigms are not directly comparable. A lack of univariate correlation between parameters does not imply incoherence; it may reflect different ways of encoding the same latent process. Conversely, apparent agreement on individual parameters may obscure inconsistencies in the overall representation. Coherence should therefore be evaluated at the level of multivariate structure rather than parameter replication. 2.3. Latent structure and multivariate coherence This study conceptualizes adoption assessments as multivariate representations of a latent adoption process that cannot be observed directly in ex ante contexts. Structured modeling and expert judgment each provide projections of this latent process into different parameter spaces. Coherence is defined as the existence of a stable association between these projections. Canonical Correlation Analysis (CCA) provides a suitable diagnostic framework for evaluating such coherence because it identifies linear combinations of variables in each representation that are maximally correlated. This allows assessment of shared structure without requiring direct correspondence between individual parameters. In this context, CCA is used to evaluate alignment between representations rather than for prediction or hypothesis testing. The magnitude and direction of the canonical correlation are interpreted as indicators of coherence at the system level. 2.4. Stability and influence in expert systems Because expert-based assessments often rely on small, purposive panels, robustness cannot be evaluated using conventional inferential statistics. Instead, the expert panel is treated as a bounded system whose properties depend on its composition. Stability is therefore assessed by examining how coherence behaves under perturbations of the expert set. Leave-one-expert-out (LOEO) analysis evaluates the influence of individual experts by re-estimating the canonical correlation after excluding each expert in turn. This identifies whether coherence is disproportionately driven by specific individuals. Complementary clustered bootstrap procedures, stratified by expert, generate alternative configurations of the expert system while preserving its internal structure. The resulting distribution of coherence measures is interpreted as a stability diagnostic rather than as an inferential confidence interval. Together, these approaches assess whether coherence is a persistent property of the expert system or sensitive to its composition. 2.5. Implications for ex ante adoption assessment By defining coherence as a multivariate and stability-dependent property, this framework shifts the evaluation of expert-based adoption assessments away from parameter comparison toward system-level alignment. Rather than asking which paradigm is more accurate, the relevant question is whether different representations provide consistent directional information about adoption dynamics when their structural differences are acknowledged. This perspective supports the joint interpretation of structured models and expert judgement as complementary representations of adoption processes. It provides a basis for integrating multiple forms of expert knowledge in decision-support contexts characterized by uncertainty and limited empirical data. 3. Methods 3.1. Study context and expert panel The empirical analysis is based on expert assessments conducted within Panama’s national agricultural research and extension system. The study focuses on a set of well-established crop production systems that have been evaluated in both ex post (Marquínez-Batista et al. 2026 ) and ex ante assessment exercises. These systems were selected to ensure that experts had sufficient experiential knowledge to provide informed assessments using both elicitation paradigms. The expert panel consisted of 14 senior professionals drawn from research, extension, and applied development roles. The panel was purposively constituted to represent the population of individuals with direct institutional responsibility for technology assessment in the selected systems. As such, the expert set constitutes a census rather than a probabilistic sample. Accordingly, the analysis focuses on characterizing properties of the expert system rather than inferring to a broader population. 3.2. Adoption assessment paradigms Each expert evaluated the same set of production systems using two elicitation approaches. First, structured assessments were conducted using the Adoption and Diffusion Outcome Prediction Tool (ADOPT). Experts responded to a standardized set of determinants related to relative advantage, learning and trialability, population heterogeneity, and contextual constraints. These responses were mapped to a parametric adoption curve, producing a multivariate representation of adoption dynamics for each expert–system combination. Second, experts provided direct judgment-based assessments of adoption outcomes, including expected adoption levels and timing, without an imposed structural model. These assessments reflect integrated expert reasoning based on contextual knowledge and experience. For analysis, outputs from both approaches were treated as alternative multivariate representations of the same underlying adoption process. 3.3. Data structure and preprocessing For each expert and production system, structured (ADOPT-based) and judgment-based outputs were organized into two sets of variables corresponding to alternative representations of adoption dynamics. Variables were screened for completeness and internal consistency prior to analysis. To ensure comparability across variables with different units and scales, all variables were standardized to a mean of zero and unit variance. This preserves the internal structure of each representation while enabling multivariate comparison. Because the objective is diagnostic rather than predictive, all variables were retained without dimensionality reduction. Observations correspond to expert–system combinations, while robustness diagnostics are interpreted with respect to the expert panel as the primary unit of analysis. 3.4. Canonical correlation analysis Canonical Correlation Analysis (CCA) was used to evaluate multivariate coherence between structured adoption model outputs and judgment-based assessments. CCA identifies linear combinations of variables within each set that maximize correlation between the two representations, thereby revealing shared structure. In this study, CCA is applied as a diagnostic tool to assess directional alignment between representations rather than for prediction or hypothesis testing. The magnitude and sign of the first canonical correlation are interpreted as indicators of system-level coherence. This interpretation follows established multivariate analysis frameworks that emphasize the role of canonical variates in identifying shared structure among complex sets of variables (Hardoon et al. 2004 ; González et al. 2008 ). 3.5. Robustness and stability diagnostics Given the expert panel's census nature, robustness is evaluated by examining how coherence responds to changes in expert composition. 3.5.1. Leave-one-expert-out (LOEO) analysis A leave-one-expert-out procedure was implemented by re-estimating the CCA after excluding each expert in turn. This assesses the influence of individual experts on the canonical correlation and identifies whether coherence depends disproportionately on specific individuals. Variation in the magnitude of the canonical correlation across LOEO iterations is interpreted as evidence of heterogeneity in expert influence rather than statistical uncertainty. 3.5.2. Clustered bootstrap by expert To further assess stability, a clustered bootstrap procedure was implemented in which experts were resampled with replacement and all associated observations retained. This preserves the internal structure of expert assessments while generating alternative configurations of the expert system. The resulting distribution of canonical correlations is interpreted as a stability diagnostic, describing how coherence varies across resampled expert compositions rather than as an inferential confidence interval. 3.6. Analytical implementation All analyses were conducted using reproducible workflows implemented in Python. Data preprocessing, matrix operations, and multivariate analyses followed standard linear algebra procedures. Graphical outputs were generated to support the interpretation of canonical relationships and robustness diagnostics. 4. Results 4.1. Univariate correspondence between elicitation paradigms Univariate relationships between parameters derived from structured adoption modeling and expert judgement are weak and inconsistent. Pairwise correlations between conceptually comparable metrics—such as adoption ceilings and adoption timing—are generally low and, in some cases, differ in sign across expert–system observations. This pattern indicates that the two elicitation paradigms do not align on individual parameters, despite being derived from the same expert system. Parameter-level comparisons, therefore, provide limited insight into agreement between representations. Table 1 reports pairwise correlations between selected ADOPT-derived parameters and their corresponding expert-judgment estimates. Table 1 Pairwise correlations between selected structured adoption parameters and expert judgement estimates. ADOPT-derived parameter Expert adoption ceiling Expert adoption timing Adoption ceiling ( \(\:{L}_{\text{ADOPT}}\) ) –0.279 –0.159 Adoption timing ( \(\:{t}_{\text{ADOPT}}\) ) 0.415 0.176 4.2. Canonical correlation analysis Canonical Correlation Analysis reveals a moderate multivariate association between structured adoption modeling and expert judgement. The first pair of canonical variates exhibits a correlation of ρ₁ = 0.47, indicating that linear combinations of variables across the two paradigms share a common underlying structure. Subsequent canonical dimensions exhibit substantially lower correlations (ρ₂ < 0.20) and are not further interpreted. This indicates that the dominant shared structure is captured by the first canonical dimension. Figure 1 presents the relationship between the first canonical variates derived from both representations, illustrating the system-level alignment captured by ρ₁. Note Points correspond to individual expert–commodity assessments. The first canonical function captures 96.1% of the shared variance explained by the canonical function. 4.3. Canonical structure coefficients To interpret the canonical relationship, we examined the structure coefficients (loadings) that link the original variables to the first canonical variates. These coefficients indicate how each variable contributes to the shared multivariate dimension. Within the structured (ADOPT-based) representation, the canonical variate is strongly associated with adoption timing (loading = 0.951), while adoption ceiling shows a moderate negative loading (–0.673). This indicates that variation along the canonical dimension is primarily driven by differences in expected diffusion timing. Within the expert judgement representation, the canonical variate is most strongly associated with adoption ceiling (loading = 0.921), with a more moderate contribution from adoption timing (loading = 0.420). This suggests that holistic assessments place greater emphasis on the expected magnitude of adoption. Taken together, these results indicate that the shared multivariate structure arises from complementary emphases across representations: structured modeling highlights temporal dynamics, while expert judgement emphasizes outcome levels. Table 2 reports canonical structure coefficients for the first canonical dimension. Table 2 Canonical structure coefficients (loadings) for the first canonical dimension. Variable set Variable Canonical loading Structured elicitation (ADOPT) Adoption ceiling –0.673 Adoption timing 0.951 Expert judgement (EXP) Adoption ceiling 0.921 Adoption timing 0.420 Note : Loadings represent correlations between the original variables and the first canonical variate of each variable set. 4.4. Leave-one-expert-out (LOEO) influence analysis Leave-one-expert-out (LOEO) analysis shows that the canonical relationship remains positive across all perturbations of the expert set, indicating that the observed multivariate association is not driven by any single individual. However, the magnitude of the canonical correlation varies across LOEO iterations, revealing heterogeneity in expert influence. Exclusion of one highly senior expert results in a substantial increase in the canonical correlation (approximately ρ ≈ 0.75), indicating that this expert’s assessments attenuate the dominant multivariate alignment. In contrast, exclusion of less influential experts produces only minor changes relative to the full-sample estimate (ρ = 0.47). Overall, the LOEO results indicate that coherence is a stable property of the expert system, while its strength varies with individual contributions. Figure 2 presents canonical correlations obtained under each LOEO iteration. Note Canonical correlation values \(\:{\rho\:}_{1}(-i)\) obtained by re-estimating the canonical correlation analysis after excluding each expert in turn. The horizontal line indicates the canonical correlation estimated using the full dataset ( \(\:{\rho\:}_{1}=0.47\) ). Values above the line indicate experts whose inclusion attenuates the dominant canonical relationship, while values below the line indicate experts whose assessments reinforce the shared multivariate structure. 4.5. Bootstrap stability diagnostics The clustered bootstrap analysis confirms the stability of the canonical relationship across alternative expert configurations. The distribution of the first canonical correlation is consistently centered above zero, indicating persistent multivariate coherence. The bootstrap distribution exhibits a bimodal pattern, with one mode near the full-sample estimate (ρ ≈ 0.47) and a second mode corresponding to higher correlation values. This reflects the presence of distinct, internally consistent configurations within the expert system. The bimodality is consistent with the LOEO results, in which the exclusion of specific influential experts leads to stronger canonical correlations. Together, these results indicate that variation in coherence magnitude is driven by differences in expert composition rather than by instability in the underlying relationship. Because the expert panel constitutes a census, the bootstrap results are interpreted as stability diagnostics rather than inferential confidence intervals. Figure 3 shows the distribution of canonical correlations across bootstrap resamples. Note Histogram showing the distribution of the first canonical correlation ( \(\:{\rho\:}_{1}\) ) obtained from bootstrap resamples in which experts were sampled with replacement while preserving their internal observations. The vertical line indicates the canonical correlation estimated using the full dataset ( \(\:{\rho\:}_{1}=0.47\) ). The bimodal distribution reflects heterogeneity in coherence strength across alternative expert configurations while maintaining a consistently positive association. 5. Discussion 5.1. Parameter mismatch as a consequence of representation The weak and inconsistent correspondence observed between structured adoption parameters and holistic expert judgement is an expected outcome of differences in how knowledge is encoded. Adoption parameters are context-dependent and shaped by how expert knowledge is elicited and structured (Feder et al. 1985 ; Doss and Morris 2001 ; Sunding and Zilberman 2001 ). Structured tools such as ADOPT decompose adoption dynamics into multiple determinants embedded in a diffusion model (Kuehne et al. 2017 ; Llewellyn et al. 2018 ). In contrast, holistic expert judgement implicitly integrates these determinants, producing outcome-level estimates without explicit decomposition. Under these conditions, parameter-level comparisons do not provide a meaningful basis for evaluating agreement. The mismatch observed in this study, therefore, reflects structural differences between representations rather than inconsistency in expert understanding. 5.2. Multivariate coherence as a criterion for agreement Despite the absence of parameter-level correspondence, the canonical correlation analysis reveals a consistent multivariate association between structured modeling and expert judgement. This supports evaluating agreement at the level of multivariate structure rather than at the level of individual variables. Adoption processes emerge from interactions among economic, institutional, biophysical, and behavioral factors. These interactions cannot be adequately captured through isolated parameters, but can be reflected in relationships among variables. Multivariate approaches such as CCA are therefore better suited to identifying whether different representations capture a shared underlying structure (Hardoon et al. 2004 ; González et al. 2008 ). The results indicate that structured modeling and expert judgment provide complementary projections of a latent adoption process. While the structured model emphasizes temporal dynamics, expert judgement places greater weight on outcome-level expectations. Agreement between paradigms should therefore be interpreted as directional alignment in system behavior rather than numerical equivalence of parameters. 5.3. Heterogeneity in expert reasoning The robustness diagnostics reveal that multivariate coherence is stable but varies in magnitude across expert configurations, indicating heterogeneity in how experts conceptualize adoption processes. This heterogeneity is consistent with the expert elicitation literature, which shows that expert judgement reflects differences in experience, disciplinary background, and institutional roles (Aspinall 2013 ; Hanea et al. 2018 ). In agricultural systems, experts may prioritize different aspects of adoption, including technological performance, farmer behavior, or institutional constraints. Rather than indicating inconsistency, this variation reflects multiple internally coherent interpretations of the system's behavior. Structured expert judgement research has long emphasized that differences across experts can reflect alternative but valid representations of uncertainty, particularly when informed by heterogeneous experience and knowledge domains (Cooke 2014 ; Hanea et al. 2018 ). Recognizing this diversity can enhance the transparency of ex ante assessments by making explicit the range of plausible perspectives that exist within the expert system. This is consistent with validation frameworks in structured expert judgement, where expert performance and influence reflect differences in calibration and information content rather than noise (Cooke 2014 ). 5.4. Expert influence and institutional memory The LOEO analysis highlights the role of highly experienced experts in shaping adoption assessments. The observed increase in canonical correlation when one senior expert is excluded indicates that some experts encode alternative representations of adoption dynamics that differ from the dominant pattern. This is consistent with the concept of institutional memory, whereby experienced experts incorporate long-term knowledge of past adoption outcomes and systemic constraints that may not be fully captured by structured models (Cooke 2014 ). These perspectives can introduce alternative but internally consistent representations within the expert system. Evaluating expert influence provides a transparent means of distinguishing between dominant consensus patterns and experience-based alternative views, enriching the interpretation of adoption dynamics in ex ante contexts. 5.5. Implications for decision support The findings have direct implications for the use of expert-based adoption assessments in research prioritization and policy design. First, parameter-level comparisons between elicitation paradigms are of limited value and may lead to misleading conclusions when structural differences are not considered. Apparent discrepancies do not necessarily indicate disagreement about adoption outcomes. Second, evaluating coherence at the multivariate level provides a more robust basis for interpreting whether different approaches offer consistent guidance. Structured models and expert judgement should therefore be used as complementary tools. Third, incorporating diagnostics such as LOEO analysis and clustered bootstrap improves transparency by revealing the stability of results and the influence of individual experts. In contexts where expert populations are limited, these diagnostics are more appropriate than conventional inferential statistics. This perspective is consistent with broader work on combining expert-based assessments, which emphasizes the importance of integrating multiple judgments while accounting for their structure and variability (Winkler et al. 2019 ). 5.6. Contribution to ex ante assessment frameworks This study contributes a diagnostic framework that reframes agreement between elicitation paradigms as a question of multivariate coherence and stability. This shifts evaluation from parameter validation toward assessment of alignment between representations. The framework is particularly relevant in data-scarce environments, where expert judgement plays a central role in decision-making. By focusing on coherence rather than parameter equivalence, it provides a more appropriate basis for interpreting expert-based assessments. More broadly, the findings support the interpretation of structured adoption tools as system-level representations of expert knowledge rather than as independent predictive models. When combined with explicit robustness diagnostics, these tools can enhance the credibility and interpretability of ex ante analyses. 5.7. Implications for systems practice Beyond agricultural adoption, the results have broader implications for systems practice in contexts where expert knowledge is expressed through multiple representations. In many complex decision environments, different analytical approaches produce outputs that appear inconsistent at the level of individual variables. A coherence-based perspective offers a way to interpret these differences by focusing on relationships among representations rather than on direct parameter comparisons. This approach is particularly relevant in settings characterized by uncertainty and limited data. Rather than seeking convergence on single estimates, analysts can evaluate whether different representations provide compatible guidance for decision-making, consistent with broader calls for transparent and user-oriented decision-support systems (Rose et al. 2021 ). Treating variation across expert assessments as informative rather than problematic allows incorporation of multiple internally coherent perspectives into more robust and transparent decision-support processes. 6. Conclusions This study examined the relationship between structured adoption modeling and holistic expert judgement in ex ante agricultural systems assessment. The results show that these approaches should be interpreted as alternative representations of expert knowledge rather than as directly comparable parameter sets. Agreement between elicitation paradigms does not emerge at the level of individual parameters but rather as a moderate, directionally stable multivariate coherence. The findings indicate that structured models and expert judgement encode adoption dynamics differently, yet remain aligned at the level of the underlying process. Structured approaches emphasize temporal diffusion dynamics through explicit decomposition of determinants, whereas holistic judgment integrates these factors into outcome-level expectations. Differences between their outputs, therefore, reflect how knowledge is represented rather than inconsistency in expert understanding. Robustness diagnostics confirm that this multivariate coherence persists across perturbations of the expert system, while revealing heterogeneity in how experts conceptualize adoption dynamics. This variation reflects multiple internally coherent representations rather than instability in the underlying relationship. Taken together, the results support the interpretation of structured adoption tools and expert judgement as complementary sources of information within a unified analytical framework. For decision-making, this implies that expert-based assessments should be evaluated in terms of coherence, stability, and transparency rather than parameter agreement. Incorporating diagnostics of expert influence and stability can further strengthen the credibility and interpretability of such assessments in data-scarce environments. More broadly, the coherence-based framework proposed here provides a practical approach for interpreting multiple representations of expert knowledge in complex decision contexts. Focusing on alignment between representations rather than direct parameter comparison enables more transparent and robust integration of expert-based evidence into research prioritization and policy design. Declarations Author Contribution All the authors' contributions are listed below:Conceptualization: Roberto QuirozMethodology: Roberto Quiroz, Liliam Marquínez-BatistaData curation: Liliam Marquínez-BatistaFormal analysis: Roberto Quiroz, Liliam Marquínez-BatistaWriting – original draft: Liliam Marquínez-BatistaWriting – review & editing: Roberto Quiroz, Jaime Espinosa-Tasón, Mariana Cruz-Chú Acknowledgement The authors thank the Government of Panama for core funding for the Instituto de Innovación Agropecuaria de Panamá (IDIAP), which supported this research. This study was conducted within the Doctoral Program at the Facultad de Ciencias Agropecuarias of the Universidad de Panamá and contributes to the first author's dissertation research. The authors also thank the following participating experts from Panama’s agricultural research and extension system for their contributions to the adoption assessment exercises: Román Gordón-Mendoza; Rodrigo A. Morales-Araúz; Ismael Camargo-Buitrago; Evelyn Quirós-Mclntire; Luis A. Barahona-Amores; Arnulfo Gutiérrez-Gutiérrez; Roberto Rodríguez-Chávez; Emigdio Rodríguez-Quiel; Francisco González-Guevara; José L. Jorge-Ramos; José A. Guerra-Murillo; and Nilso García. Data Availability The data supporting the findings of this study are available from the corresponding author upon reasonable request. References Aspinall WP, Cooke RM (2013) Quantifying scientific uncertainty from expert judgement elicitation. In: Risk and Uncertainty Assessment for Natural Hazards (eds. Rougier, J., Sparks, S. & Hill, L.J.) 64–99 https://doi.org/10.1017/CBO9781139047562.005 Burke WJ, Snapp SS, Jayne TS (2020) An in-depth analysis of maize yield response to fertilizer in Central Malawi reveals low profits and too many weeds. Agric Econ 51:923–940. https://doi.org/10.1111/agec.12601 Cooke RM (2014) Validation of expert judgment with the classical model. In: Martini C, Boumans M (eds) Experts and consensus in social science. Springer, Cham, pp 147–160. https://doi.org/10.1007/978-3-319-08551-7_10 Doss CR, Morris ML (2001) How does gender affect the adoption of agricultural innovations? The case of improved maize technology in Ghana. Agric Econ 25:27–39. https://doi.org/10.1111/j.1574-0862.2001.tb00233.x Feder G, Just RE, Zilberman D (1985) Adoption of agricultural innovations in developing countries: A survey. Econ Dev Cult Change 33:255–298. https://doi.org/10.1086/451461 González I, Déjean S, Martin PGP, Baccini A (2008) CCA: An R package to extend canonical correlation analysis. J Stat Softw 23:1–14. https://doi.org/10.18637/jss.v023.i12 Hanea AM, McBride MF, Burgman MA, Wintle BC, Fidler F, Flander L, Twardy C, Manning B (2017) Investigate–Discuss–Estimate–Aggregate for structured expert judgement. Int J Forecast 33:267–279. https://doi.org/10.1016/j.ijforecast.2016.02.008 Hanea AM, Burgman MA, Hemming V, McBride MF, Wintle BC (2018) The value of performance weights and discussion in aggregated expert judgments. Risk Anal 38:1781–1794. https://doi.org/10.1111/risa.12992 Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: An overview with application to learning methods. Neural Comput 16:2639–2664. https://doi.org/10.1162/0899766042321814 Kuehne G, Llewellyn R, Pannell DJ, Wilkinson R, Dolling P, Ouzman J, Ewing M (2017) Predicting farmer uptake of new agricultural practices: A tool for research, development and extension planning. Agric Syst 156:115–125. https://doi.org/10.1016/j.agsy.2017.06.007 Llewellyn RS, Kuehne G, Pannell DJ, Wilkinson R, Dolling P, Ouzman J, Ewing M (2018) Predicting the adoption of innovations by farmers: What is different in practice? Agric Syst 163:123–135. https://doi.org/10.1002/aepp.1301 Marquínez-Batista LM, Espinosa-Tasón J, Cruz-Chu M, Hertentains-Caballero L, Gordón-Mendoza R, Morales-Araúz RA, Quiroz R (2026) Investigación agropecuaria, una inversión rentable: Impacto económico de tecnologías de IDIAP en seis rubros productivos. Ciencia Agropecuaria 42:192–221. http://www.revistacienciaagropecuaria.ac.pa/index.php/ciencia-agropecuaria/article/view/699 Prokopy LS, Floress K, Arbuckle JG, Church SP, Eanes FR, Gao Y, Gramig BM, Ranjan P, Singh AS (2019) Adoption of agricultural conservation practices in the United States: Evidence from 35 years of quantitative literature. J Soil Water Conserv 74:520–534. https://doi.org/10.2489/jswc.74.5.520 Rose DC, Wheeler R, Winter M, Lobley M, Chivers CA (2021) Agriculture 4.0: Making it work for people, production and the planet. Land Use Policy 100:104933. https://doi.org/10.1016/j.landusepol.2020.104933 Ruzzante S, Labarta R, Bilton A (2021) Adoption of agricultural technology in developing countries: A meta-analysis of the literature. World Dev 146:105606. https://doi.org/10.1016/j.worlddev.2021.105599 Sunding D, Zilberman D (2001) The agricultural innovation process: Research and technology adoption in a changing agricultural sector. In: Gardner B, Rausser G (eds) Handbook of agricultural economics, vol 1A. Elsevier, Amsterdam, pp 207–261. https://doi.org/10.1016/S1574-0072(01)10007-1 Winkler RL, Grushka-Cockayne Y, Lichtendahl KC, Jose VRR (2019) Probability forecasts and their combination: A research perspective. Decis Anal 16:239–260. https://doi.org/10.1287/deca.2019.0391 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9383379","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":634062017,"identity":"b19a8013-7e98-4d58-a6ad-46e621a6c41a","order_by":0,"name":"Liliam Marquínez-Batista","email":"","orcid":"","institution":"Instituto de Innovación Agropecuaria de Panamá","correspondingAuthor":false,"prefix":"","firstName":"Liliam","middleName":"","lastName":"Marquínez-Batista","suffix":""},{"id":634062020,"identity":"1b723ca2-5ff6-495c-b0ea-cccce27556fb","order_by":1,"name":"Jaime Espinosa-Tasón","email":"","orcid":"","institution":"Instituto de Innovación Agropecuaria de Panamá","correspondingAuthor":false,"prefix":"","firstName":"Jaime","middleName":"","lastName":"Espinosa-Tasón","suffix":""},{"id":634062021,"identity":"428da6fa-8a28-40fb-9a33-dbdc9ad00c88","order_by":2,"name":"Mariana Cruz-Chú","email":"","orcid":"","institution":"National University of San Marcos","correspondingAuthor":false,"prefix":"","firstName":"Mariana","middleName":"","lastName":"Cruz-Chú","suffix":""},{"id":634062024,"identity":"c0ae29b6-ac7f-49df-9519-1b5d1b1e6cbf","order_by":3,"name":"Roberto Quiroz","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBUlEQVRIie3SMWrDMBSA4ScC6SLiVYZiX0FeksX4LM8I3KUXKHRwEThL09mlQ8/QGygYlMW4ayBLQy6gLiWFEOIE3GRw4o6F6h/Ee6APLQKw2f5ktBlIWh9hM3QR/CEJQP/35FBRkw4wGk+mq7t1CIO5yIzZvPvOq3z4gPsInBfVSq7LSgQlJuDOY/mc00WQayI5aAGswlbC2O3QTbEAXpMeZQsETTIGfQWctoo9GX0fCa/Q12S8hu1FMiRHggp5/QqQ7AKhlXDTJKFuuZQkVyJ407Fk8ZOgrDxDribTzzQMvcHsZgVmE/leMVsa8xV5zmM7aaKgTlc8+RXnU503bDab7b+2Axq1V/YCYDeKAAAAAElFTkSuQmCC","orcid":"","institution":"Instituto de Innovación Agropecuaria de Panamá","correspondingAuthor":true,"prefix":"","firstName":"Roberto","middleName":"","lastName":"Quiroz","suffix":""}],"badges":[],"createdAt":"2026-04-10 23:08:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9383379/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9383379/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":108978236,"identity":"816ac75c-777a-4857-8f5e-8baf2f9b639e","added_by":"auto","created_at":"2026-05-11 11:35:18","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":88323,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eCanonical scores for the first canonical variates derived from ADOPT-based adoption parameters and expert judgement estimates.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNote: Points correspond to individual expert–commodity assessments. The first canonical function captures 96.1% of the shared variance explained by the canonical function.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-9383379/v1/66cde794e5dedd699469c167.png"},{"id":108954931,"identity":"82725236-6353-4138-a9d1-648eefa83a4b","added_by":"auto","created_at":"2026-05-11 08:00:33","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":64790,"visible":true,"origin":"","legend":"\u003cp\u003eLeave-one-expert-out influence on the first canonical correlation.\u003c/p\u003e\n\u003cp\u003eNote: Canonical correlation values ρ\u003csub\u003e1\u003c/sub\u003e (-i) obtained by re-estimating the canonical correlation analysis after excluding each expert in turn. The horizontal line indicates the canonical correlation estimated using the full dataset (ρ\u003csub\u003e1\u003c/sub\u003e=0.47). Values above the line indicate experts whose inclusion attenuates the dominant canonical relationship, while values below the line indicate experts whose assessments reinforce the shared multivariate structure.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-9383379/v1/21103acc362d50dec89f93c0.png"},{"id":108954959,"identity":"3eb4466a-df9f-42d7-b9da-e550cbe2d98e","added_by":"auto","created_at":"2026-05-11 08:00:43","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":73608,"visible":true,"origin":"","legend":"\u003cp\u003eClustered bootstrap distribution of the first canonical correlation.\u003c/p\u003e\n\u003cp\u003eNote: Histogram showing the distribution of the first canonical correlation (ρ\u003csub\u003e1\u003c/sub\u003e) obtained from bootstrap resamples in which experts were sampled with replacement while preserving their internal observations. The vertical line indicates the canonical correlation estimated using the full dataset (ρ\u003csub\u003e1\u003c/sub\u003e=0.47). The bimodal distribution reflects heterogeneity in coherence strength across alternative expert configurations while maintaining a consistently positive association.\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-9383379/v1/31475d26d3d9b372ac4e32de.png"},{"id":108979831,"identity":"517912b7-a13e-40b1-ba8d-c9d2c629f595","added_by":"auto","created_at":"2026-05-11 12:01:47","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":454967,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9383379/v1/ee3618f7-2e5c-4508-bdbd-1232aab33647.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A systems-based coherence framework for interpreting expert knowledge in ex ante adoption assessment","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eEx ante assessment of agricultural technology adoption plays a central role in research prioritization, extension planning, and policy design. In complex socio-technical systems, such assessments rely heavily on expert knowledge to anticipate system behavior under uncertainty, particularly when empirical data are unavailable or incomplete. Because adoption processes unfold over long time horizons, decision-making frequently depends on how expert knowledge is elicited, structured, and interpreted across different analytical approaches (Feder et al. \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e1985\u003c/span\u003e; Prokopy et al. \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Burke et al. \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2020\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eA substantial body of empirical research has examined the determinants of agricultural technology adoption using farm-level data and econometric methods. These studies show that adoption is shaped by interacting economic, biophysical, social, and institutional factors, and that estimated effects are highly context-dependent (Doss and Morris \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2001\u003c/span\u003e; Sunding and Zilberman \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2001\u003c/span\u003e). More recent syntheses further highlight that drivers of adoption are sensitive to how adoption processes are conceptualized and measured, complicating comparisons across studies and analytical tools (Prokopy et al. \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; Ruzzante et al. \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eIn ex ante contexts, where longitudinal adoption data are unavailable, expert elicitation has become an essential source of information for anticipating adoption dynamics (Aspinall \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Hanea et al. \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2017\u003c/span\u003e). However, elicitation paradigms differ in how they structure expert knowledge, raising questions about how their outputs should be interpreted when applied to the same problem. Structured approaches formalize knowledge through predefined determinants embedded in analytical models, whereas holistic approaches elicit direct judgements that integrate contextual and experiential knowledge without imposing explicit structure.\u003c/p\u003e \u003cp\u003eThe Adoption and Diffusion Outcome Prediction Tool (ADOPT) is a widely used structured approach that translates expert assessments of adoption determinants into parametric diffusion trajectories (Kuehne et al. \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Llewellyn et al. \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). In contrast, holistic expert judgement provides direct estimates of adoption outcomes without explicit decomposition of underlying drivers. These approaches encode expert knowledge differently, making direct comparison at the level of individual parameters conceptually problematic. Structured models distribute information across multiple determinants, whereas holistic judgements compress interacting factors into a smaller set of outcome estimates.\u003c/p\u003e \u003cp\u003eExisting comparisons between these paradigms often assume that agreement should emerge at the level of individual parameters, such as adoption ceilings or timing. This assumption may be misleading because it ignores structural differences in how adoption dynamics are represented. Apparent discrepancies at the parameter level may therefore reflect differences in representation rather than disagreement about the underlying process.\u003c/p\u003e \u003cp\u003eThis study addresses this issue by proposing a systems-based coherence framework for interpreting expert-based adoption assessments in ex ante contexts. Rather than testing for parameter equivalence, we evaluate whether structured adoption modeling and holistic expert judgement exhibit a stable multivariate association that reflects a shared latent conception of adoption dynamics. Canonical Correlation Analysis (CCA) is used as a diagnostic tool to assess alignment between representations, complemented by leave-one-expert-out (LOEO) influence analysis and clustered bootstrap resampling interpreted as stability diagnostics (Hardoon et al. \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2004\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe empirical analysis is based on a census of 14 senior experts from Panama\u0026rsquo;s agricultural research and extension system who evaluated established production systems using both elicitation paradigms. Because the expert panel constitutes a bounded population rather than a probabilistic sample, robustness is assessed through diagnostics of stability and influence rather than inferential statistics (Aspinall \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Hanea et al. \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eBy reframing agreement between elicitation paradigms as a question of multivariate coherence and stability, this study provides a practical basis for interpreting expert-based adoption assessments in decision-support contexts characterized by uncertainty and limited data.\u003c/p\u003e"},{"header":"2. Conceptual framework","content":"\u003cp\u003eEx ante assessment of agricultural technology adoption requires translating expert knowledge into quantitative representations to inform research prioritization and policy decisions. In systems terms, these representations can be understood as alternative ways of encoding knowledge about the behavior of a complex socio-technical system. Because these representations differ in structure and level of aggregation, they may not be directly comparable at the level of individual parameters. The framework developed here distinguishes between structured adoption modeling and holistic expert judgement and defines their coherence as a system-level property.\u003c/p\u003e \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1. Alternative encodings of expert knowledge\u003c/h2\u003e \u003cp\u003eStructured adoption models decompose adoption dynamics into predefined determinants that jointly shape diffusion processes. From a systems perspective, this approach makes explicit the internal structure of the representation by separating interacting components. Tools such as ADOPT operationalize this paradigm by eliciting standardized assessments of dimensions such as relative advantage, learning requirements, population heterogeneity, and contextual constraints, which are then mapped onto a parametric adoption curve. Adoption outcomes emerge from the interaction among these determinants within a coherent model structure.\u003c/p\u003e \u003cp\u003eIn contrast, holistic expert judgement elicits direct assessments of adoption outcomes\u0026mdash;such as expected adoption ceilings or timing\u0026mdash;without imposing an explicit internal structure. Experts integrate contextual knowledge, experience, and tacit understanding, but the weighting of underlying determinants remains implicit. This results in a compressed representation in which interactions are embedded but not explicitly parameterized.\u003c/p\u003e \u003cp\u003eBoth paradigms aim to represent the same underlying phenomenon: the expected trajectory of technology adoption within a given system. Rather than constituting independent sources of information, they can be interpreted as alternative encodings of a shared latent conception of adoption dynamics. Differences between their outputs, therefore, reflect how knowledge is structured rather than necessarily indicating disagreement.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2. Coherence versus parameter correspondence\u003c/h2\u003e \u003cp\u003eEvaluations of expert-based adoption assessments often assume that coherence should be reflected in correspondence between individual parameters, such as similar estimates of adoption ceilings or time to widespread uptake. This assumption is problematic because it ignores structural differences between representations.\u003c/p\u003e \u003cp\u003eAdoption is inherently multivariate, shaped by interacting determinants that jointly influence diffusion trajectories. Structured models distribute this information across multiple parameters, whereas holistic judgements integrate these interactions into fewer outcome estimates. As a result, parameters across paradigms are not directly comparable.\u003c/p\u003e \u003cp\u003eA lack of univariate correlation between parameters does not imply incoherence; it may reflect different ways of encoding the same latent process. Conversely, apparent agreement on individual parameters may obscure inconsistencies in the overall representation. Coherence should therefore be evaluated at the level of multivariate structure rather than parameter replication.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3. Latent structure and multivariate coherence\u003c/h2\u003e \u003cp\u003eThis study conceptualizes adoption assessments as multivariate representations of a latent adoption process that cannot be observed directly in ex ante contexts. Structured modeling and expert judgment each provide projections of this latent process into different parameter spaces. Coherence is defined as the existence of a stable association between these projections.\u003c/p\u003e \u003cp\u003eCanonical Correlation Analysis (CCA) provides a suitable diagnostic framework for evaluating such coherence because it identifies linear combinations of variables in each representation that are maximally correlated. This allows assessment of shared structure without requiring direct correspondence between individual parameters.\u003c/p\u003e \u003cp\u003eIn this context, CCA is used to evaluate alignment between representations rather than for prediction or hypothesis testing. The magnitude and direction of the canonical correlation are interpreted as indicators of coherence at the system level.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003e2.4. Stability and influence in expert systems\u003c/h2\u003e \u003cp\u003eBecause expert-based assessments often rely on small, purposive panels, robustness cannot be evaluated using conventional inferential statistics. Instead, the expert panel is treated as a bounded system whose properties depend on its composition. Stability is therefore assessed by examining how coherence behaves under perturbations of the expert set.\u003c/p\u003e \u003cp\u003eLeave-one-expert-out (LOEO) analysis evaluates the influence of individual experts by re-estimating the canonical correlation after excluding each expert in turn. This identifies whether coherence is disproportionately driven by specific individuals.\u003c/p\u003e \u003cp\u003eComplementary clustered bootstrap procedures, stratified by expert, generate alternative configurations of the expert system while preserving its internal structure. The resulting distribution of coherence measures is interpreted as a stability diagnostic rather than as an inferential confidence interval.\u003c/p\u003e \u003cp\u003eTogether, these approaches assess whether coherence is a persistent property of the expert system or sensitive to its composition.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e2.5. Implications for ex ante adoption assessment\u003c/h2\u003e \u003cp\u003eBy defining coherence as a multivariate and stability-dependent property, this framework shifts the evaluation of expert-based adoption assessments away from parameter comparison toward system-level alignment. Rather than asking which paradigm is more accurate, the relevant question is whether different representations provide consistent directional information about adoption dynamics when their structural differences are acknowledged.\u003c/p\u003e \u003cp\u003eThis perspective supports the joint interpretation of structured models and expert judgement as complementary representations of adoption processes. It provides a basis for integrating multiple forms of expert knowledge in decision-support contexts characterized by uncertainty and limited empirical data.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Methods","content":"\u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.1. Study context and expert panel\u003c/h2\u003e \u003cp\u003eThe empirical analysis is based on expert assessments conducted within Panama\u0026rsquo;s national agricultural research and extension system. The study focuses on a set of well-established crop production systems that have been evaluated in both ex post (Marqu\u0026iacute;nez-Batista et al. \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2026\u003c/span\u003e) and ex ante assessment exercises. These systems were selected to ensure that experts had sufficient experiential knowledge to provide informed assessments using both elicitation paradigms.\u003c/p\u003e \u003cp\u003eThe expert panel consisted of 14 senior professionals drawn from research, extension, and applied development roles. The panel was purposively constituted to represent the population of individuals with direct institutional responsibility for technology assessment in the selected systems. As such, the expert set constitutes a census rather than a probabilistic sample. Accordingly, the analysis focuses on characterizing properties of the expert system rather than inferring to a broader population.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e3.2. Adoption assessment paradigms\u003c/h2\u003e \u003cp\u003eEach expert evaluated the same set of production systems using two elicitation approaches. First, structured assessments were conducted using the Adoption and Diffusion Outcome Prediction Tool (ADOPT). Experts responded to a standardized set of determinants related to relative advantage, learning and trialability, population heterogeneity, and contextual constraints. These responses were mapped to a parametric adoption curve, producing a multivariate representation of adoption dynamics for each expert\u0026ndash;system combination.\u003c/p\u003e \u003cp\u003eSecond, experts provided direct judgment-based assessments of adoption outcomes, including expected adoption levels and timing, without an imposed structural model. These assessments reflect integrated expert reasoning based on contextual knowledge and experience.\u003c/p\u003e \u003cp\u003eFor analysis, outputs from both approaches were treated as alternative multivariate representations of the same underlying adoption process.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e3.3. Data structure and preprocessing\u003c/h2\u003e \u003cp\u003eFor each expert and production system, structured (ADOPT-based) and judgment-based outputs were organized into two sets of variables corresponding to alternative representations of adoption dynamics. Variables were screened for completeness and internal consistency prior to analysis.\u003c/p\u003e \u003cp\u003eTo ensure comparability across variables with different units and scales, all variables were standardized to a mean of zero and unit variance. This preserves the internal structure of each representation while enabling multivariate comparison.\u003c/p\u003e \u003cp\u003eBecause the objective is diagnostic rather than predictive, all variables were retained without dimensionality reduction. Observations correspond to expert\u0026ndash;system combinations, while robustness diagnostics are interpreted with respect to the expert panel as the primary unit of analysis.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e3.4. Canonical correlation analysis\u003c/h2\u003e \u003cp\u003eCanonical Correlation Analysis (CCA) was used to evaluate multivariate coherence between structured adoption model outputs and judgment-based assessments. CCA identifies linear combinations of variables within each set that maximize correlation between the two representations, thereby revealing shared structure.\u003c/p\u003e \u003cp\u003eIn this study, CCA is applied as a diagnostic tool to assess directional alignment between representations rather than for prediction or hypothesis testing. The magnitude and sign of the first canonical correlation are interpreted as indicators of system-level coherence. This interpretation follows established multivariate analysis frameworks that emphasize the role of canonical variates in identifying shared structure among complex sets of variables (Hardoon et al. \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2004\u003c/span\u003e; Gonz\u0026aacute;lez et al. \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2008\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e3.5. Robustness and stability diagnostics\u003c/h2\u003e \u003cp\u003eGiven the expert panel's census nature, robustness is evaluated by examining how coherence responds to changes in expert composition.\u003c/p\u003e \u003cdiv id=\"Sec14\" class=\"Section3\"\u003e \u003ch2\u003e3.5.1. Leave-one-expert-out (LOEO) analysis\u003c/h2\u003e \u003cp\u003eA leave-one-expert-out procedure was implemented by re-estimating the CCA after excluding each expert in turn. This assesses the influence of individual experts on the canonical correlation and identifies whether coherence depends disproportionately on specific individuals.\u003c/p\u003e \u003cp\u003eVariation in the magnitude of the canonical correlation across LOEO iterations is interpreted as evidence of heterogeneity in expert influence rather than statistical uncertainty.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec15\" class=\"Section3\"\u003e \u003ch2\u003e3.5.2. Clustered bootstrap by expert\u003c/h2\u003e \u003cp\u003eTo further assess stability, a clustered bootstrap procedure was implemented in which experts were resampled with replacement and all associated observations retained. This preserves the internal structure of expert assessments while generating alternative configurations of the expert system.\u003c/p\u003e \u003cp\u003eThe resulting distribution of canonical correlations is interpreted as a stability diagnostic, describing how coherence varies across resampled expert compositions rather than as an inferential confidence interval.\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003e3.6. Analytical implementation\u003c/h2\u003e \u003cp\u003eAll analyses were conducted using reproducible workflows implemented in Python. Data preprocessing, matrix operations, and multivariate analyses followed standard linear algebra procedures. Graphical outputs were generated to support the interpretation of canonical relationships and robustness diagnostics.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Results","content":"\u003cdiv id=\"Sec18\" class=\"Section2\"\u003e \u003ch2\u003e4.1. Univariate correspondence between elicitation paradigms\u003c/h2\u003e \u003cp\u003eUnivariate relationships between parameters derived from structured adoption modeling and expert judgement are weak and inconsistent. Pairwise correlations between conceptually comparable metrics\u0026mdash;such as adoption ceilings and adoption timing\u0026mdash;are generally low and, in some cases, differ in sign across expert\u0026ndash;system observations.\u003c/p\u003e \u003cp\u003eThis pattern indicates that the two elicitation paradigms do not align on individual parameters, despite being derived from the same expert system. Parameter-level comparisons, therefore, provide limited insight into agreement between representations.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e reports pairwise correlations between selected ADOPT-derived parameters and their corresponding expert-judgment estimates.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003ePairwise correlations between selected structured adoption parameters and expert judgement estimates.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eADOPT-derived parameter\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eExpert adoption ceiling\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eExpert adoption timing\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdoption ceiling ( \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{L}_{\\text{ADOPT}}\\)\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u0026ndash;0.279\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u0026ndash;0.159\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAdoption timing ( \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{t}_{\\text{ADOPT}}\\)\u003c/span\u003e\u003c/span\u003e)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.415\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.176\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec19\" class=\"Section2\"\u003e \u003ch2\u003e4.2. Canonical correlation analysis\u003c/h2\u003e \u003cp\u003eCanonical Correlation Analysis reveals a moderate multivariate association between structured adoption modeling and expert judgement. The first pair of canonical variates exhibits a correlation of ρ₁ = 0.47, indicating that linear combinations of variables across the two paradigms share a common underlying structure.\u003c/p\u003e \u003cp\u003eSubsequent canonical dimensions exhibit substantially lower correlations (ρ₂ \u0026lt; 0.20) and are not further interpreted. This indicates that the dominant shared structure is captured by the first canonical dimension.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e presents the relationship between the first canonical variates derived from both representations, illustrating the system-level alignment captured by ρ₁.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003ePoints correspond to individual expert\u0026ndash;commodity assessments. The first canonical function captures 96.1% of the shared variance explained by the canonical function.\u003c/p\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003ch2\u003e4.3. Canonical structure coefficients\u003c/h2\u003e \u003cp\u003eTo interpret the canonical relationship, we examined the structure coefficients (loadings) that link the original variables to the first canonical variates. These coefficients indicate how each variable contributes to the shared multivariate dimension.\u003c/p\u003e \u003cp\u003eWithin the structured (ADOPT-based) representation, the canonical variate is strongly associated with adoption timing (loading\u0026thinsp;=\u0026thinsp;0.951), while adoption ceiling shows a moderate negative loading (\u0026ndash;0.673). This indicates that variation along the canonical dimension is primarily driven by differences in expected diffusion timing.\u003c/p\u003e \u003cp\u003eWithin the expert judgement representation, the canonical variate is most strongly associated with adoption ceiling (loading\u0026thinsp;=\u0026thinsp;0.921), with a more moderate contribution from adoption timing (loading\u0026thinsp;=\u0026thinsp;0.420). This suggests that holistic assessments place greater emphasis on the expected magnitude of adoption.\u003c/p\u003e \u003cp\u003eTaken together, these results indicate that the shared multivariate structure arises from complementary emphases across representations: structured modeling highlights temporal dynamics, while expert judgement emphasizes outcome levels. Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e reports canonical structure coefficients for the first canonical dimension.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eCanonical structure coefficients (loadings) for the first canonical dimension.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eVariable set\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eVariable\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCanonical loading\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eStructured elicitation (ADOPT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdoption ceiling\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u0026ndash;0.673\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdoption timing\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.951\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eExpert judgement (EXP)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdoption ceiling\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.921\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAdoption timing\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.420\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"3\"\u003e\u003cb\u003eNote\u003c/b\u003e: Loadings represent correlations between the original variables and the first canonical variate of each variable set.\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003e4.4. Leave-one-expert-out (LOEO) influence analysis\u003c/h2\u003e \u003cp\u003eLeave-one-expert-out (LOEO) analysis shows that the canonical relationship remains positive across all perturbations of the expert set, indicating that the observed multivariate association is not driven by any single individual.\u003c/p\u003e \u003cp\u003eHowever, the magnitude of the canonical correlation varies across LOEO iterations, revealing heterogeneity in expert influence. Exclusion of one highly senior expert results in a substantial increase in the canonical correlation (approximately ρ\u0026thinsp;\u0026asymp;\u0026thinsp;0.75), indicating that this expert\u0026rsquo;s assessments attenuate the dominant multivariate alignment. In contrast, exclusion of less influential experts produces only minor changes relative to the full-sample estimate (ρ\u0026thinsp;=\u0026thinsp;0.47).\u003c/p\u003e \u003cp\u003eOverall, the LOEO results indicate that coherence is a stable property of the expert system, while its strength varies with individual contributions. Figure\u0026nbsp;\u003cspan refid=\"Fig2\" class=\"InternalRef\"\u003e2\u003c/span\u003e presents canonical correlations obtained under each LOEO iteration.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003eCanonical correlation values \u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\rho\\:}_{1}(-i)\\)\u003c/span\u003e\u003c/span\u003e obtained by re-estimating the canonical correlation analysis after excluding each expert in turn. The horizontal line indicates the canonical correlation estimated using the full dataset (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\rho\\:}_{1}=0.47\\)\u003c/span\u003e\u003c/span\u003e). Values above the line indicate experts whose inclusion attenuates the dominant canonical relationship, while values below the line indicate experts whose assessments reinforce the shared multivariate structure.\u003c/p\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003e4.5. Bootstrap stability diagnostics\u003c/h2\u003e \u003cp\u003eThe clustered bootstrap analysis confirms the stability of the canonical relationship across alternative expert configurations. The distribution of the first canonical correlation is consistently centered above zero, indicating persistent multivariate coherence.\u003c/p\u003e \u003cp\u003eThe bootstrap distribution exhibits a bimodal pattern, with one mode near the full-sample estimate (ρ\u0026thinsp;\u0026asymp;\u0026thinsp;0.47) and a second mode corresponding to higher correlation values. This reflects the presence of distinct, internally consistent configurations within the expert system.\u003c/p\u003e \u003cp\u003eThe bimodality is consistent with the LOEO results, in which the exclusion of specific influential experts leads to stronger canonical correlations. Together, these results indicate that variation in coherence magnitude is driven by differences in expert composition rather than by instability in the underlying relationship.\u003c/p\u003e \u003cp\u003eBecause the expert panel constitutes a census, the bootstrap results are interpreted as stability diagnostics rather than inferential confidence intervals. Figure\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e shows the distribution of canonical correlations across bootstrap resamples.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003e \u003cstrong\u003eNote\u003c/strong\u003e \u003cp\u003eHistogram showing the distribution of the first canonical correlation (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\rho\\:}_{1}\\)\u003c/span\u003e\u003c/span\u003e) obtained from bootstrap resamples in which experts were sampled with replacement while preserving their internal observations. The vertical line indicates the canonical correlation estimated using the full dataset (\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:{\\rho\\:}_{1}=0.47\\)\u003c/span\u003e\u003c/span\u003e). The bimodal distribution reflects heterogeneity in coherence strength across alternative expert configurations while maintaining a consistently positive association.\u003c/p\u003e \u003c/p\u003e \u003c/div\u003e"},{"header":"5. Discussion","content":"\u003cdiv id=\"Sec24\" class=\"Section2\"\u003e \u003ch2\u003e5.1. Parameter mismatch as a consequence of representation\u003c/h2\u003e \u003cp\u003eThe weak and inconsistent correspondence observed between structured adoption parameters and holistic expert judgement is an expected outcome of differences in how knowledge is encoded. Adoption parameters are context-dependent and shaped by how expert knowledge is elicited and structured (Feder et al. \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e1985\u003c/span\u003e; Doss and Morris \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2001\u003c/span\u003e; Sunding and Zilberman \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2001\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eStructured tools such as ADOPT decompose adoption dynamics into multiple determinants embedded in a diffusion model (Kuehne et al. \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2017\u003c/span\u003e; Llewellyn et al. \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). In contrast, holistic expert judgement implicitly integrates these determinants, producing outcome-level estimates without explicit decomposition. Under these conditions, parameter-level comparisons do not provide a meaningful basis for evaluating agreement.\u003c/p\u003e \u003cp\u003eThe mismatch observed in this study, therefore, reflects structural differences between representations rather than inconsistency in expert understanding.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec25\" class=\"Section2\"\u003e \u003ch2\u003e5.2. Multivariate coherence as a criterion for agreement\u003c/h2\u003e \u003cp\u003eDespite the absence of parameter-level correspondence, the canonical correlation analysis reveals a consistent multivariate association between structured modeling and expert judgement. This supports evaluating agreement at the level of multivariate structure rather than at the level of individual variables.\u003c/p\u003e \u003cp\u003eAdoption processes emerge from interactions among economic, institutional, biophysical, and behavioral factors. These interactions cannot be adequately captured through isolated parameters, but can be reflected in relationships among variables. Multivariate approaches such as CCA are therefore better suited to identifying whether different representations capture a shared underlying structure (Hardoon et al. \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2004\u003c/span\u003e; Gonz\u0026aacute;lez et al. \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2008\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThe results indicate that structured modeling and expert judgment provide complementary projections of a latent adoption process. While the structured model emphasizes temporal dynamics, expert judgement places greater weight on outcome-level expectations. Agreement between paradigms should therefore be interpreted as directional alignment in system behavior rather than numerical equivalence of parameters.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec26\" class=\"Section2\"\u003e \u003ch2\u003e5.3. Heterogeneity in expert reasoning\u003c/h2\u003e \u003cp\u003eThe robustness diagnostics reveal that multivariate coherence is stable but varies in magnitude across expert configurations, indicating heterogeneity in how experts conceptualize adoption processes.\u003c/p\u003e \u003cp\u003eThis heterogeneity is consistent with the expert elicitation literature, which shows that expert judgement reflects differences in experience, disciplinary background, and institutional roles (Aspinall \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2013\u003c/span\u003e; Hanea et al. \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). In agricultural systems, experts may prioritize different aspects of adoption, including technological performance, farmer behavior, or institutional constraints.\u003c/p\u003e \u003cp\u003eRather than indicating inconsistency, this variation reflects multiple internally coherent interpretations of the system's behavior. Structured expert judgement research has long emphasized that differences across experts can reflect alternative but valid representations of uncertainty, particularly when informed by heterogeneous experience and knowledge domains (Cooke \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Hanea et al. \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). Recognizing this diversity can enhance the transparency of ex ante assessments by making explicit the range of plausible perspectives that exist within the expert system. This is consistent with validation frameworks in structured expert judgement, where expert performance and influence reflect differences in calibration and information content rather than noise (Cooke \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2014\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec27\" class=\"Section2\"\u003e \u003ch2\u003e5.4. Expert influence and institutional memory\u003c/h2\u003e \u003cp\u003eThe LOEO analysis highlights the role of highly experienced experts in shaping adoption assessments. The observed increase in canonical correlation when one senior expert is excluded indicates that some experts encode alternative representations of adoption dynamics that differ from the dominant pattern.\u003c/p\u003e \u003cp\u003eThis is consistent with the concept of institutional memory, whereby experienced experts incorporate long-term knowledge of past adoption outcomes and systemic constraints that may not be fully captured by structured models (Cooke \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2014\u003c/span\u003e). These perspectives can introduce alternative but internally consistent representations within the expert system.\u003c/p\u003e \u003cp\u003eEvaluating expert influence provides a transparent means of distinguishing between dominant consensus patterns and experience-based alternative views, enriching the interpretation of adoption dynamics in ex ante contexts.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec28\" class=\"Section2\"\u003e \u003ch2\u003e5.5. Implications for decision support\u003c/h2\u003e \u003cp\u003eThe findings have direct implications for the use of expert-based adoption assessments in research prioritization and policy design.\u003c/p\u003e \u003cp\u003eFirst, parameter-level comparisons between elicitation paradigms are of limited value and may lead to misleading conclusions when structural differences are not considered. Apparent discrepancies do not necessarily indicate disagreement about adoption outcomes.\u003c/p\u003e \u003cp\u003eSecond, evaluating coherence at the multivariate level provides a more robust basis for interpreting whether different approaches offer consistent guidance. Structured models and expert judgement should therefore be used as complementary tools.\u003c/p\u003e \u003cp\u003eThird, incorporating diagnostics such as LOEO analysis and clustered bootstrap improves transparency by revealing the stability of results and the influence of individual experts. In contexts where expert populations are limited, these diagnostics are more appropriate than conventional inferential statistics. This perspective is consistent with broader work on combining expert-based assessments, which emphasizes the importance of integrating multiple judgments while accounting for their structure and variability (Winkler et al. \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e2019\u003c/span\u003e).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec29\" class=\"Section2\"\u003e \u003ch2\u003e5.6. Contribution to ex ante assessment frameworks\u003c/h2\u003e \u003cp\u003eThis study contributes a diagnostic framework that reframes agreement between elicitation paradigms as a question of multivariate coherence and stability. This shifts evaluation from parameter validation toward assessment of alignment between representations.\u003c/p\u003e \u003cp\u003eThe framework is particularly relevant in data-scarce environments, where expert judgement plays a central role in decision-making. By focusing on coherence rather than parameter equivalence, it provides a more appropriate basis for interpreting expert-based assessments.\u003c/p\u003e \u003cp\u003eMore broadly, the findings support the interpretation of structured adoption tools as system-level representations of expert knowledge rather than as independent predictive models. When combined with explicit robustness diagnostics, these tools can enhance the credibility and interpretability of ex ante analyses.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec30\" class=\"Section2\"\u003e \u003ch2\u003e5.7. Implications for systems practice\u003c/h2\u003e \u003cp\u003eBeyond agricultural adoption, the results have broader implications for systems practice in contexts where expert knowledge is expressed through multiple representations.\u003c/p\u003e \u003cp\u003eIn many complex decision environments, different analytical approaches produce outputs that appear inconsistent at the level of individual variables. A coherence-based perspective offers a way to interpret these differences by focusing on relationships among representations rather than on direct parameter comparisons.\u003c/p\u003e \u003cp\u003eThis approach is particularly relevant in settings characterized by uncertainty and limited data. Rather than seeking convergence on single estimates, analysts can evaluate whether different representations provide compatible guidance for decision-making, consistent with broader calls for transparent and user-oriented decision-support systems (Rose et al. \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2021\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eTreating variation across expert assessments as informative rather than problematic allows incorporation of multiple internally coherent perspectives into more robust and transparent decision-support processes.\u003c/p\u003e \u003c/div\u003e"},{"header":"6. Conclusions","content":"\u003cp\u003eThis study examined the relationship between structured adoption modeling and holistic expert judgement in ex ante agricultural systems assessment. The results show that these approaches should be interpreted as alternative representations of expert knowledge rather than as directly comparable parameter sets. Agreement between elicitation paradigms does not emerge at the level of individual parameters but rather as a moderate, directionally stable multivariate coherence.\u003c/p\u003e \u003cp\u003eThe findings indicate that structured models and expert judgement encode adoption dynamics differently, yet remain aligned at the level of the underlying process. Structured approaches emphasize temporal diffusion dynamics through explicit decomposition of determinants, whereas holistic judgment integrates these factors into outcome-level expectations. Differences between their outputs, therefore, reflect how knowledge is represented rather than inconsistency in expert understanding.\u003c/p\u003e \u003cp\u003eRobustness diagnostics confirm that this multivariate coherence persists across perturbations of the expert system, while revealing heterogeneity in how experts conceptualize adoption dynamics. This variation reflects multiple internally coherent representations rather than instability in the underlying relationship.\u003c/p\u003e \u003cp\u003eTaken together, the results support the interpretation of structured adoption tools and expert judgement as complementary sources of information within a unified analytical framework. For decision-making, this implies that expert-based assessments should be evaluated in terms of coherence, stability, and transparency rather than parameter agreement. Incorporating diagnostics of expert influence and stability can further strengthen the credibility and interpretability of such assessments in data-scarce environments.\u003c/p\u003e \u003cp\u003eMore broadly, the coherence-based framework proposed here provides a practical approach for interpreting multiple representations of expert knowledge in complex decision contexts. Focusing on alignment between representations rather than direct parameter comparison enables more transparent and robust integration of expert-based evidence into research prioritization and policy design.\u003c/p\u003e"},{"header":"Declarations","content":"\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eAll the authors' contributions are listed below:Conceptualization: Roberto QuirozMethodology: Roberto Quiroz, Liliam Marqu\u0026iacute;nez-BatistaData curation: Liliam Marqu\u0026iacute;nez-BatistaFormal analysis: Roberto Quiroz, Liliam Marqu\u0026iacute;nez-BatistaWriting \u0026ndash; original draft: Liliam Marqu\u0026iacute;nez-BatistaWriting \u0026ndash; review \u0026amp;amp; editing: Roberto Quiroz, Jaime Espinosa-Tas\u0026oacute;n, Mariana Cruz-Ch\u0026uacute;\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eThe authors thank the Government of Panama for core funding for the Instituto de Innovaci\u0026oacute;n Agropecuaria de Panam\u0026aacute; (IDIAP), which supported this research. This study was conducted within the Doctoral Program at the Facultad de Ciencias Agropecuarias of the Universidad de Panam\u0026aacute; and contributes to the first author's dissertation research. The authors also thank the following participating experts from Panama\u0026rsquo;s agricultural research and extension system for their contributions to the adoption assessment exercises: Rom\u0026aacute;n Gord\u0026oacute;n-Mendoza; Rodrigo A. Morales-Ara\u0026uacute;z; Ismael Camargo-Buitrago; Evelyn Quir\u0026oacute;s-Mclntire; Luis A. Barahona-Amores; Arnulfo Guti\u0026eacute;rrez-Guti\u0026eacute;rrez; Roberto Rodr\u0026iacute;guez-Ch\u0026aacute;vez; Emigdio Rodr\u0026iacute;guez-Quiel; Francisco Gonz\u0026aacute;lez-Guevara; Jos\u0026eacute; L. Jorge-Ramos; Jos\u0026eacute; A. Guerra-Murillo; and Nilso Garc\u0026iacute;a.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eThe data supporting the findings of this study are available from the corresponding author upon reasonable request.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eAspinall WP, Cooke RM (2013) Quantifying scientific uncertainty from expert judgement elicitation. In: \u003cem\u003eRisk and Uncertainty Assessment for Natural Hazards\u003c/em\u003e (eds. Rougier, J., Sparks, S. \u003cstrong\u003e\u0026amp;\u003c/strong\u003e Hill, L.J.) 64\u0026ndash;99 https://doi.org/10.1017/CBO9781139047562.005\u003c/li\u003e\n\u003cli\u003eBurke WJ, Snapp SS, Jayne TS (2020) An in-depth analysis of maize yield response to fertilizer in Central Malawi reveals low profits and too many weeds. Agric Econ 51:923\u0026ndash;940. https://doi.org/10.1111/agec.12601\u003c/li\u003e\n\u003cli\u003eCooke RM (2014) Validation of expert judgment with the classical model. In: Martini C, Boumans M (eds) Experts and consensus in social science. Springer, Cham, pp 147\u0026ndash;160. https://doi.org/10.1007/978-3-319-08551-7_10\u003c/li\u003e\n\u003cli\u003eDoss CR, Morris ML (2001) How does gender affect the adoption of agricultural innovations? The case of improved maize technology in Ghana. Agric Econ 25:27\u0026ndash;39. https://doi.org/10.1111/j.1574-0862.2001.tb00233.x\u003c/li\u003e\n\u003cli\u003eFeder G, Just RE, Zilberman D (1985) Adoption of agricultural innovations in developing countries: A survey. Econ Dev Cult Change 33:255\u0026ndash;298. https://doi.org/10.1086/451461\u003c/li\u003e\n\u003cli\u003eGonz\u0026aacute;lez I, D\u0026eacute;jean S, Martin PGP, Baccini A (2008) CCA: An R package to extend canonical correlation analysis. J Stat Softw 23:1\u0026ndash;14. https://doi.org/10.18637/jss.v023.i12\u003c/li\u003e\n\u003cli\u003eHanea AM, McBride MF, Burgman MA, Wintle BC, Fidler F, Flander L, Twardy C, Manning B (2017) Investigate\u0026ndash;Discuss\u0026ndash;Estimate\u0026ndash;Aggregate for structured expert judgement. Int J Forecast 33:267\u0026ndash;279. https://doi.org/10.1016/j.ijforecast.2016.02.008\u003c/li\u003e\n\u003cli\u003eHanea AM, Burgman MA, Hemming V, McBride MF, Wintle BC (2018) The value of performance weights and discussion in aggregated expert judgments. Risk Anal 38:1781\u0026ndash;1794. https://doi.org/10.1111/risa.12992\u003c/li\u003e\n\u003cli\u003eHardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: An overview with application to learning methods. Neural Comput 16:2639\u0026ndash;2664. https://doi.org/10.1162/0899766042321814\u003c/li\u003e\n\u003cli\u003eKuehne G, Llewellyn R, Pannell DJ, Wilkinson R, Dolling P, Ouzman J, Ewing M (2017) Predicting farmer uptake of new agricultural practices: A tool for research, development and extension planning. Agric Syst 156:115\u0026ndash;125. https://doi.org/10.1016/j.agsy.2017.06.007\u003c/li\u003e\n\u003cli\u003eLlewellyn RS, Kuehne G, Pannell DJ, Wilkinson R, Dolling P, Ouzman J, Ewing M (2018) Predicting the adoption of innovations by farmers: What is different in practice? Agric Syst 163:123\u0026ndash;135. https://doi.org/10.1002/aepp.1301\u003c/li\u003e\n\u003cli\u003eMarqu\u0026iacute;nez-Batista LM, Espinosa-Tas\u0026oacute;n J, Cruz-Chu M, Hertentains-Caballero L, Gord\u0026oacute;n-Mendoza R, Morales-Ara\u0026uacute;z RA, Quiroz R (2026) Investigaci\u0026oacute;n agropecuaria, una inversi\u0026oacute;n rentable: Impacto econ\u0026oacute;mico de tecnolog\u0026iacute;as de IDIAP en seis rubros productivos. Ciencia Agropecuaria 42:192\u0026ndash;221. http://www.revistacienciaagropecuaria.ac.pa/index.php/ciencia-agropecuaria/article/view/699\u003c/li\u003e\n\u003cli\u003eProkopy LS, Floress K, Arbuckle JG, Church SP, Eanes FR, Gao Y, Gramig BM, Ranjan P, Singh AS (2019) Adoption of agricultural conservation practices in the United States: Evidence from 35 years of quantitative literature. J Soil Water Conserv 74:520\u0026ndash;534. https://doi.org/10.2489/jswc.74.5.520\u003c/li\u003e\n\u003cli\u003eRose DC, Wheeler R, Winter M, Lobley M, Chivers CA (2021) Agriculture 4.0: Making it work for people, production and the planet. Land Use Policy 100:104933. https://doi.org/10.1016/j.landusepol.2020.104933\u003c/li\u003e\n\u003cli\u003eRuzzante S, Labarta R, Bilton A (2021) Adoption of agricultural technology in developing countries: A meta-analysis of the literature. World Dev 146:105606. https://doi.org/10.1016/j.worlddev.2021.105599\u003c/li\u003e\n\u003cli\u003eSunding D, Zilberman D (2001) The agricultural innovation process: Research and technology adoption in a changing agricultural sector. In: Gardner B, Rausser G (eds) Handbook of agricultural economics, vol 1A. Elsevier, Amsterdam, pp 207\u0026ndash;261. https://doi.org/10.1016/S1574-0072(01)10007-1\u003c/li\u003e\n\u003cli\u003eWinkler RL, Grushka-Cockayne Y, Lichtendahl KC, Jose VRR (2019) Probability forecasts and their combination: A research perspective. Decis Anal 16:239\u0026ndash;260. https://doi.org/10.1287/deca.2019.0391\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"technology adoption, expert elicitation, ADOPT, canonical correlation analysis, ex ante assessment, decision support","lastPublishedDoi":"10.21203/rs.3.rs-9383379/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9383379/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eAccurate interpretation of expert-based assessments is critical for decision-making in complex socio-technical systems, particularly in ex ante contexts where empirical data are unavailable. In such settings, expert knowledge is often expressed through multiple representations that differ in structure, transparency, and level of aggregation. However, the relationships among these alternative representations remain poorly understood, limiting their effective use in decision support. This study proposes a systems-based coherence framework to interpret structured adoption modeling and holistic expert judgment as alternative encodings of a shared latent adoption process. Fourteen senior experts from Panama\u0026rsquo;s agricultural research and extension system evaluated established production systems using both the Adoption and Diffusion Outcome Prediction Tool (ADOPT) and direct judgment-based assessments. Multivariate alignment between representations was evaluated using Canonical Correlation Analysis as a diagnostic tool, complemented by leave-one-expert-out influence analysis and clustered bootstrap resampling interpreted as stability diagnostics. Results show weak and inconsistent correspondence at the level of individual parameters, but a moderate, directionally stable multivariate association (ρ\u0026thinsp;=\u0026thinsp;0.47) between the representations. Robustness analyses indicate that this coherence persists across alternative expert configurations, despite heterogeneity in individual influence. These findings suggest that agreement between expert-based assessments should be evaluated as a system-level property rather than through parameter equivalence. The proposed framework provides a practical basis for interpreting expert-based assessments in decision-support contexts characterized by uncertainty and limited data.\u003c/p\u003e","manuscriptTitle":"A systems-based coherence framework for interpreting expert knowledge in ex ante adoption assessment","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-05-11 07:59:46","doi":"10.21203/rs.3.rs-9383379/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a975bfaf-bb68-41d1-86e3-9e38b0914193","owner":[],"postedDate":"May 11th, 2026","published":true,"recentEditorialEvents":[{"type":"reviewerAgreed","content":"114623887835454071307052809387043843198","date":"2026-05-01T10:04:18+00:00","index":12,"fulltext":""},{"type":"reviewersInvited","content":"4","date":"2026-05-01T01:56:29+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2026-05-11T07:59:46+00:00","versionOfRecord":[],"versionCreatedAt":"2026-05-11 07:59:46","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9383379","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9383379","identity":"rs-9383379","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}