Known unknowns and model selection in ecological evidence synthesis

Abstract Quantitative evidence synthesis is a prominent path towards generality in ecology. Generality is typically discussed in terms of central tendencies, such as an average effect across a compilation of studies, and the role of heterogeneity for assessing generality is not as well developed. Heterogeneity examines the transferability of ecological effects across contexts, though between-study variance is typically assumed as constant (i.e., homoscedastic). Here, I use two case studies to show how relaxing the assumption of homoscedasticity and cross validation can combine to further the goals of evidence syntheses. First, I examine scale-dependent heterogeneity for a meta-analysis of plant native-exotic species richness relationships, quantifying the relationships among unexplained effect size variation, spatial grain and extent. Second, I examine relationships among patch size, study-level covariates and unexplained variation in species richness using a database of fragmentation studies. Heteroscedastic models quantify where effects can be transferred with more or less certainty, and provide new descriptions of transferability for both case studies. Cross validation can be applied to a single or multiple models, adapted to either the goal of intervention or generalization, and showed that assuming homoscedasticity can limit transferability for both case studies. Competing Interest Statement The authors have declared no competing interest. Footnotes Open research: Data used in this work are available from Peng et al. (2019) https://doi.org/10.5061/dryad.59kv753, and Chase et al. (2019) http://onlinelibrary.wiley.com/doi/10.1002/ecy.2861/suppinfo. Code to reproduce results are available at: https://github.com/sablowes/heterogeneity-evidence-synthesis, and will be archived upon acceptance. This version has been revised to make the key points come through more clearly.