The statistical fragility of animal cognition findings: a meta-meta-analytic reappraisal

This is a Preprint and has not been peer reviewed. This is version 1 of this Preprint. You must log in to post a comment. There are no comments or no comments have been made public for this article. This is a Preprint and has not been peer reviewed. This is version 1 of this Preprint. Add a Comment You must log in to post a comment. Comments There are no comments or no comments have been made public for this article. How reliable is the evidence in animal cognition research? Concerns are mounting over the statistical robustness of this and other fields. Many primary studies rely on small samples and rarely report null results, while meta-analyses sometimes overlook publication bias, all of which may contribute to unreliable conclusions. We conducted a second-order meta-analysis across 28 published meta-analytical papers in the animal cognition field to evaluate three inferential metrics: statistical power, Type M (magnitude) error, and Type S (sign) error. These were calculated at the primary study and meta-analysis levels. To approximate the true effect, we used the mean effect size from each meta-analysis; when publication bias was detected, we applied a correction to mitigate potential overestimation. Our results indicate low statistical power and inflated effect sizes in both primary studies and meta-analyses. After bias correction, on average, power decreased from 17% to 9% and effect size values decreased from 82% to 45%. Type M errors were common, indicating that statistically significant results often exaggerated the underlying effect sizes. For improving the reliability of animal cognition research, we recommend preregistering and transparently reporting both primary and secondary studies. We also call for the routine application of publication bias correction in meta-analytical syntheses. https://doi.org/10.32942/X2Z36M Comparative and Evolutionary Physiology, Comparative Psychology, Ecology and Evolutionary Biology, Psychology Comparative Psychology, evidence synthesis, meta-research, Quantitative synthesis, replication crisis, Research methodology Published: 2025-11-06 17:52 Last Updated: 2025-11-06 17:52 CC BY Attribution 4.0 International Conflict of interest statement: NA Data and Code Availability Statement: The data and scripts we used are available at https://github.com/Ayumi-495/statistical_power_AnimCogn. Language: English