{"paper_id":"3dd5377f-28cf-41f2-b13d-3c8bee33918b","body_text":"Evaluating the applicability of replication success\nmetrics in animal-to-human translation: A simulation\nstudy\nCarolyne Jie Huang 1,2, Samuel Pawel 3, Kimberley Elaine Wever 4, Benjamin Victor\nIneichen5, and Rachel Heyard 6*\n1Master Program in Biostatistics, Epidemiology, Biostatistics and Prevention Institute, University of Zurich,\nZurich, Switzerland\n2Newborn Research, Department of Neonatology, University and University Hospital Zurich, Zurich, Switzerland\n3Department of Biostatistics, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich,\nSwitzerland\n4Department of Anesthesiology, Pain and Palliative Medicine, Radboud university medical center, Nijmegen,\nThe Netherlands\n5Department of Clinical Research, University of Bern, Bern, Switzerland\n6Center for Reproducible Science and Research Synthesis, Epidemiology, Biostatistics and Prevention Institute,\nUniversity of Zurich, Zurich, Switzerland\n*Corresponding author: Rachel Heyard, rachel.heyard@uzh.ch\nNovember 7, 2025\nThis is a preprint which has not yet been peer reviewed.\n1\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \nNOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice.\n\nAbstract\nTranslation failure, in which promising animal study results can not be reproduced in human\ntrials, is a challenge in biomedical research. Metrics for replication success are widely used to evaluate\nreproducibility, i.e., the extent to which the results of a study agree with those of replication studies.\nThe relevance of these metrics in assessing animal-to-human translation success (or faillure) is unclear.\nWe conducted a simulation study to examine whether these metrics can quantify translation success\nand how their performance varies under different conditions. Using parameters from a meta-analysis\non prenatal amino acid supplementation and maternal blood pressure, we simulated animal and\nhumanstudiesunder648scenarios, varyingeffectsizes, heterogeneity, animalsamplesizesandnumber\nof pooled animal studies. Nine metrics were assessed, namely the two-trials rule, meta-analysis,\nreplication Bayes factor, unweighted and weighted Edgington’s methods, golden scepticalp-value\nand three versions of controlled scepticalp-value. Most metrics, except meta-analysis and replication\nBayesfactor, controlledfalsepositiveratesundernoheterogeneity, butbecameliberalasheterogeneity\nincreased, particularlybetweenhumanstudies. Translationpower(i.e., theprobabilityoftruepositive\ntranslation success) was constrained by the weaker evidence of the two findings; e.g., small sample\nsize in the animal studies resulted in lower translation power. The metric based on meta-analysis\nfrequently indicated success when either of the species found strong evidence, while scepticalp-values\nwere more conservative. The scepticalp-value that controls overall type-one error and the weighted\nversion of Edgington’s method performed relatively consistently across scenarios. No metric was\nuniformly optimal. Metrics developed for replication studies can inform assessments of translation,\nbut their utility depends on the underlying evidence and assumptions. Using multiple metrics in\ncombination, with attention to their strengths and limitations, is recommended for evaluating the\ntranslation of animal findings to human outcomes.\nKeywords: Reproducibility, translatability, generalizability, meta-research, metrics, simulation study,\nanimal study, human study\n2\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n1 Introduction\nIn many fields of medical research, drugs that show promising results in preclinical studies in animals\nfrequently fail to do the same in human clinical trials [1]. This “translation failure” is one of the biggest\nchallenges in biomedical research today: it is estimated that around two-thirds to 95% of therapeutics\nfound to be safe and effective during animal testing, fail when tested in humans [1, 2, 3, 4]. Translatability\nhasbeendefinedas“theabilitytoapplyresearchdiscoveriesfromexperimentalmodelstoapplicationsthat\ndirectly benefit humans” [5]. Reasons for low translatability are multifaceted. However, the pervasiveness\nof suboptimal study design, analysis, and reporting, potentially resulting in a lack of reproducibility (i.e.,\nthe “extent to which the results of a study agree with those of replication studies” [5]), has been flagged\nas a key concern [2, 6]. Animal studies often demonstrate deficiencies such as inaccurate or inconsistent\ndata collection procedures, poor reporting of key variables, including the age and sex of animals used,\nand a lack of measures to reduce risks of bias, including the absence of randomization or blinding [7, 8,\n9]. In terms of statistical methodology, frequent issues include small sample sizes, leading to low powered\nstudies, inadequate control for confounding variables, and insufficient description of statistical methods\nor reporting of uncertainty measures [10, 11, 12, 13]. Publication bias, the phenomenon in which the\ndecision to publish a study is based on the direction or strength of its findings, is also rampant. Animal\nstudies reporting positive and statistically significant results are more likely to be published than those\nwith negative or statistically non-significant findings, meaning that subsequent human studies may be\nbased upon biased conclusions [14, 15]. Such issues are a detriment to both animals, whose lives are\nwasted when we draw incorrect conclusions from research performed on them; and humans, who are put\nat unnecessary risk during clinical trials when an intervention’s reported safety or efficacy is overstated\nor outright false [7, 16].\nReplication and translation Recently, there has been a growing interest in replications of previously\npublished studies. A replication is defined as a “study that repeats all or part of another study and\nallows researchers to compare their findings” [5]. To perform a replication study, researchers could for\nexample use the same methodology and/or analysis as presented in an original study on newly collected\ndata. They then attempt to determine if the results from the replication study are consistent with\nthose in the original study [17]. A multitude of metrics have been used or proposed to quantify the\nconsistency of results, and ultimately to decide if a replication was “successful” or not [18]. We will\nrefer to these metrics as “replication success metrics”. They might compare, for example, thep-values or\nthe magnitude, direction, or uncertainty of estimated treatment effects obtained from the original and\nreplication studies. Other metrics, such as the one based on a meta-analysis, combine results from an\noriginal study and its replication attempt(s) to estimate an overall effect size [19, 20, 21]. So far, studies\nattempting to estimate how often translation failure occurs have largely utilized the simple statistical\nsignificance criterion, i.e., assessing if the animal and human studies both report a statistically significant\ntreatment effect in the same direction, often referred to as the two-trials rule [22]. To our knowledge,\nthe usage of alternative replication success metrics in a translation setting has not yet been investigated.\nTranslationcontrastswithreplicationinthatanimalandhumanstudiesexaminedifferentpopulationsand\noften have different experimental designs, and thus inherently produce different results. As such, a human\nstudy is a “conceptual” rather than a “direct” replication of the animal study [23]. As a result, metrics\nwhich are useful for measuring replication may not be as applicable for translation. This distinction\nmotivates the current simulation study.\nStudy objectives To date, little is known about the most appropriate metrics to assess or quantify\n“translatability” or “translation success”. In this study, we aim to investigate whether metrics proposed\n3\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nto quantify replication success can be applied, and are useful, in the context of the translation of results\nfrom animals to humans. We will investigate the ability of these metrics to quantify translation success\nunder various simulation conditions, for example in different scenarios of effect sizes and sample sizes, in\norder to gain a better understanding of their behaviour and characteristics.\n2 Methods\n2.1 Study design, data, and protocol\nThis is a simulation study. Synthetic data sets, representing animal studies and human studies, were\ngenerated according to various simulation conditions (see Sections 2.3, 2.4 and 2.5). The simulated\nanimal and human findings were then used to evaluate the selected translation success metrics (pre-\nsented in Section 2.7) using pre-specified performance measures (Section 2.8). A detailed protocol of\nthe present simulation study, following the ADEMP (Aims, Data-generating mechanisms, Estimands and\nother targets, Methods, Performance measure) preregistration template [24], was preregistered on the\nOpen Science Framework prior to running the simulation study [25].\n2.2 Protocol amendments\nDuring drafting of this manuscript, we found a conceptual error in our data generating mechanism.\nInitially, weplannedtoincorporatetheheterogeneityvariancedirectlyintothesimulationoftheindividual\nobservations for the animal, respectively the human, study. We now first use the heterogeneity variance\nto simulate an effect size for the animal, respectively the human, study, and then use this effect size\nand solely the sampling variation to simulate the individual observations for the animal, respectively\nthe human, study. Finally, a coding error in the protocol (in the calculation of the pooled variance the\n“-2” was missing in the denominator, page 4 of the protocol) led to the wrong human group sample size\nshowing up in the protocol text (103 instead of the correct 107).\n2.3 Data generation\nWe assumed that the synthetic studies investigate the effect of a treatment (e.g., prenatal amino acid\nsupplementation) on an outcome that is comparable across animals and humans (e.g., maternal blood\npressure as in Terstappen et al. [26]). We simulated individual observations of this outcome measurement\nfor the animals, respectively for the humans, in the treatment group and the control group. To generate\nthe synthetic data, the true animal and human means in the treatment groups were set toµA and µH.\nThe mean in the animal and human control groups was always set to 0, so thatµA and µH correspond to\nmean difference effect sizes. The true effect size heterogeneity variances were set toτ 2\nA and τ 2\nH. For each\nsimulation repetition i we performed the following:\n1. Simulation of effect sizes:We first simulatedk animal effect sizes and one human effect size:\nµA,i,j ∼ N(µA, τ 2\nA), j = 1, . . . , k,\nµH,i ∼ N(µH, τ 2\nH).\n2. Simulation of the animal finding: We generated k synthetic animal studies. Each study j\n(j = 1, . . . , k) had nA animals in the control group (C) andnA animals in the treatment group (T).\nThe outcomes were simulated as\n4\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nyA,C,i,j,l ∼ N(0, s2\nA) and yA,T,i,j,l ∼ N(µA,i,j, s2\nA),\nwith l = 1, . . . , nA. For each of thek synthetic animal studies we then performed a one-sided two-\nsample t-test, comparing the outcomes for the treatment group to the control group. Thek effect\nestimates were pooled using a random-effects meta-analysis (with restricted maximum likelihood\nestimator for heterogeneity variance) and the resulting pooled effect size, standard error andp-value\nconstituted the “animal finding”.\n3. Simulation of the human finding:We simulated outcome measurements for the human study,\nwith nH humans in the control group (C) andnH humans in the treatment group (T):\nyH,C,i,m ∼ N(0, s2\nH) and yH,T,i,m ∼ N(µH,i, s2\nH),\nwhere m = 1, . . . , nH. A one-sided two-samplet-test was then performed on the simulated human\noutcome measurements, comparing treatment and control group, and the resulting effect size (mean\ndifference), standard error of the mean difference andp-value were extracted. This constitutes the\n“human finding”.\nWe performed one-sided tests as they take into account the direction of the effect estimate and we were\ntesting for a “beneficial treatment effect”. Note that we use the one-sided tests with significance level\nα = 0.025, which is equivalent to performing two-sided tests at level0.05 and checking that the effect\ngoes in the beneficial direction.\n2.4 Motivating dataset\nThe selection of the values of the simulation parameters in Section 2.3 was based on a systematic review\nand meta-analysis by Terstappen et al. [26] (also Figure 2.1 in Huang and Heyard [25]), which assessed the\neffects of prenatal amino acid supplementation on birth weight and, as a secondary outcome, maternal\nblood pressure (BP). In this simulation study, we focused only on the blood pressure data, for which\nmeasures from animal and human subjects were comparable. From this data we extracted the species-\nspecific random-effects meta-analytical treatment effects, θA and θH, and the estimated heterogeneity\nvariances T 2\nA and T 2\nH. The typical within-study variances, s2\nA and s2\nH, were also computed. The data\nincluded 15 animal studies and six human studies with an average sample sizes per group of 8.7 for the\nanimal studies and 26.8 for the human studies.\n2.5 Simulation conditions\nTo investigate the applicability of the replication success metrics in the context of animal to human trans-\nlation, we simulated the animal and human findings under various conditions (i.e., values for parameters\nin Section 2.3). The conditions were chosen based on previous literature [7, 13] and expert knowledge,\nwith the aim of emulating plausible real-world translation scenarios as closely as possible.\nAnimal and human effect sizes,µA and µH In the motivating dataset, we found thatθA = −24.37\nand θH = −4.44, meaning that the beneficial treatment effect (a reduction in blood pressure) found in\nanimals was larger than the beneficial treatment effect in humans. However, this might not always be the\ncase and the true treatment effect in animals and humans might be the same, i.e., both small, both large,\nor both absent entirely. The true effect size in humans could also be larger than in animals. We therefore\nsimulated under all possible combinations of animal and human effect sizes, summarized in Table 1.\n5\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAnimal and human study heterogeneity τ 2\nA and τ 2\nH We implemented a similar setup for the\nbetween-study heterogeneity variances. In our motivating dataset, animal studies had a higher degree\nof heterogeneity (T 2\nA = 291.1) than human studies (T 2\nH = 19.5). Again, we simulated under all possible\ncombinations of small (i.e., T 2\nH), large (i.e., T 2\nA), or zero animal and human study heterogeneity to\ninvestigate the behaviour of translation metrics across different scenarios (see Table 1).\nAnimal and human study sample sizes,nA and nH Animal preclinical studies commonly suffer\nfrom insufficient sample sizes, leading to underpowered studies [2, 14, 6]. Therefore, we included two\ndifferent sample sizes per group in each of the simulated animal studies. The typical sample size observed\nin animal studies is small, with approximately 10 animals per group [26, 10, 27]. To investigate the effect\nof an increased sample size on translation success, we simulated with a larger sample size of 20 animals\nper group. This represents the maximum number of animals per group observed in our data example\nfrom Terstappen et al. [26] and in Table 1 of Hooijmans et al. [27]. For the human findings we used a\nfixed samples size and always simulated withnH = 107 humans per group. This sample size was chosen\nto reach 80% power for an absolute effect of|θH| = 4.44 and typical within-study variance s2\nH with a\none-sided two-sample t-test and significance levelα = 0.025.\nNumber of pooled animal studies,k In real-world drug development, multiple animal studies are\nusually performed and results are pooled before deciding to progress to a human study. We investigated\nthe effect of pooling together different numbers of animal studies on translation success:k = 2, 3, 4,\nand 5. As described above, we performed a random-effects meta-analysis (using the restricted maximum\nlikelihood estimator for the heterogeneity variance) of thek animal studies to generate the animal finding.\nWe varied the factors above and summarized in Table 1 in a fully factorial manner. This resulted in 3\n(animal effect size) × (human effect size) × 3 (animal heterogeneity) × 3 (human heterogeneity) × 2\n(animal sample size)× 4 (number of animal studies to pool together,k) = 648 simulation conditions.\nTable 1: Summary of the simulation factors used to generate animal and human studies (varied in a fully\nfactorial way).\nParameter Value Description\nµA {0, −4.44, −24.37} True animal effect size\nµH {0, −4.44, −24.37} True human effect size\nτ 2\nA {0, 19.5, 291.1} Heterogeneity across animal studies\nτ 2\nH {0, 19.5, 291.1} Heterogeneity across human studies\nnA {10, 20} Group sample size for animal studies (small or larger)\nk {2, 3, 4, 5} Number of animal studies to pool together\n2.6 Criteria to continue from an animal study to a human study\nUsually, animal studies must show evidence of a beneficial treatment effect before a treatment is tested\nin humans. Alternatively, treatments that show no evidence for a beneficial effect in animal studies may\ncontinue to testing in humans if the treatment is safe and its mechanism of action is plausible in humans\n[28, 29]. When analysing the applicability of translation success metrics, we considered both of these\ncontinuation criteria and excluded the human studies accordingly.\n1. Strict criterion– Here, we only “moved on” to a human study, if the random-effects meta-analysis\nof the correspondingk animal studies found a significant beneficial treatment effect with one-sided\n6\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nsignificance level α = 0.025.\n2. Lenient criterion– Under this more lenient criterion, we “moved on” to a human study when-\never the random-effects meta-analysis of thek animal studies found a beneficial effect (i.e., if the\nestimated effect was negative), even if it was not statistically significant atα = 0.025.\n3. No criterion– As a point of reference, we also computed performance measures for all simulated\nanimal and human findings, regardless of the results of the animal studies.\n2.7 Translation success metrics\nWe compared the characteristics of nine translation success metrics, including the replication success\nmetrics used in Freuli, Held, and Heyard [19] and Muradchanian et al. [20] as well as some more re-\ncently developed metrics, across the previously defined simulation conditions: the significance criterion,\nthe meta-analysis, the replication Bayes factor, the unweighted and the weighted Edgington method,\nthe controlled scepticalp-value and three versions of the golden scepticalp-value. These metrics were\nprimarily designed to assess replication success in the pairwise comparison of one original study with one\nreplication study. In the current translation setting, the result of the random-effects meta-analysis of the\nk animal studies was treated as the “animal finding”.\nSignificance criterion (Two-trials rule) The significance criterion, often referred to as the two-\ntrials rule, is the current standard for a new drug to meet prior to its approval. It requires that two\nindependent studies demonstrate a drug’s efficacy at a certain significance level, usuallyα = 0.025 for\none-sided hypothesis testing [22] to take into account direction of effect. This criterion is also often used\nto identify replication success in large-scale reproducibility projects [18]. According to the significance\ncriterion, we flagged a successful translation if both the animal and human studies yielded evidence for\na beneficial treatment effect, both at a significance level ofα = 0.025 [20]:\nmax{pA, pH} ≤ α = 0.025,\nwhere pA and pH represent thep-values found in the animal and human studies, respectively. By setting\nα = 0.025, the significance criterion controls the overall T1E rate, or in our case the rate of a false positive\ntranslation success, atα2 = 0.0252 = 0.000625 [22]. Note that this metric treats the animal and human\nfinding as interchangeable.\nMeta-analysis According to the meta-analysis criterion, we flagged translation success if a fixed-effects\nmeta-analysis combining the animal and the human findings found a significant effect in the desired direc-\ntion (here, a decrease), at a one-sided significance levelα2, i.e., pMA < α 2. This threshold again ensured\nan overall T1E control atα2 [30]. We followed Freuli, Held, and Heyard [19] for the implementation\nof the method via the weighted version of Stouffer’s method described in Cousins [31], and define the\nmeta-analytic p-value pMA of a one-sided test for a negative effect (i.e., a beneficial effect) as follows:\npMA = Φ\n \nσHzA + σAzH\np\nσ2\nA + σ2\nH\n!\n,\nwhere Φ(·) is the standard normal cumulative distribution function, and zA and zH are the z-values\nrepresenting the findings in the synthetic animal studies (pooled) and human study. If we were to test\nfor a positive effect, the formula for the desiredp-value would change to 1 − Φ(. . .). Note that we\nused fixed-effects meta-analysis as this represents the commonly used metric in the replication context.\n7\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAlternatively, we could have used random-effects meta-analysis, but the assessment of heterogeneity is\nchallenging when only two findings (animal and human) are considered [32]. The metric based on meta-\nanalysis treats the animal and human finding as interchangeable.\nReplication Bayes factor In the translation setting, the replication Bayes factor (BF) quantifies the\nevidence that the outcome observed in a human study is absent or spurious (H0) relative to the evidence\nthat the outcome in the human study is consistent with that found in the (original) animal studies (Hr)\n[33]. To calculate the replication BF, BF0r, Hr is defined as the alternative hypothesis that the human\neffect is distributed according to the posterior distribution of the effect after observing the animal finding.\nA translation was flagged as successful if\nBF0r < 1/3,\nusing the conventional threshold for substantial evidence forHr over H0 [34].\nUnweighted & weighted Edgington’s methodEdgington [35] developed an additive method of\ncombining p-values from independent experiments, which has been applied more recently in a replication\nsuccess setting [36]. Under the original version of Edgington’s method, to control the overall T1E rate\nacross two studies at levelα2 = 0.0252, a successful replication is flagged if the sum of thep-values in the\noriginal study po and the replication studypr is smaller than\n√\n2 · α ≈ 0.035. With Edgington’s method,\nit is possible to flag success even if one ofpo or pr is not significant, as long aspo + pr ≤ 0.035. In our\nstudy, a successful translation was flagged with Edgington’s method if\npA + pH ≤\n√\n2 · α ≈ 0.035.\nEven more recently, a weighted version of Edgington’s method has been proposed [36]. Here, an original\nstudy is down-weighted and a replication study is up-weighted to account for potential biases in the\noriginal study, and both study findings are not interchangeable any more. For the same overall T1E\ncontrol at levelα2 = 0.0252, and in the case in which a replication study carries twice the weight of the\noriginal study, a successful replication is flagged ifpo + 2pr ≤ 2 · α = 0.05 [36]. In our study, we gave the\nhuman result more weight than the animal finding, and flagged a successful translation with weighted\nEdgington’s method if\npA + 2pH ≤ 2 · α = 0.05.\nGolden & controlled scepticalp-value The scepticalp-value combines a reverse-Bayes technique\nwith a prior-data conflict assessment. The extent to which the data in a replication or human study con-\nflicts with a sceptical prior which renders the original or animal finding not convincing, can be quantified\nwith the scepticalp-value [37, 20]. In our study, we will examine two specific versions of the sceptical\np-value: the golden and controlled scepticalp-value [38, 39].\nWith the golden sceptical p-value ps, success can be flagged if the p-value of the animal finding is\nsufficiently small (i.e.,pA ≈ α), even if it is not necessarily significant, as long as there is no shrinkage\nin effect size in the human study [38, 19]. However, in the translation setting, shrinkage of effect size is\nexpected in human studies relative to animal studies. Held, Micheloud, and Pawel [38] have developed a\nmethod to calculate a threshold˜α for the golden scepticalp-value in the presence of shrinkage, which is\nequivalent to a pre-specified threshold ofα when no shrinkage is present. The golden scepticalp-value\ncontrols the overall T1E rate at a maximum level ofα2, provided that the sample size of the replication\nor human study is larger than in the original or animal study. We can therefore calculate the values for\n˜α below which a successful translation can be flagged even ifpA is sightly higher thenα, in the presence\n8\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nof no (0%) shrinkage, moderate (25%) shrinkage, and high (50%) shrinkage. Note that these levels of\nshrinkage were selected rather arbitrarily, based on an observed shrinkage of about 50% in the Replication\nProject Psychology [40], while it was even higher in the Replication Project Cancer Biology (mean effect\nsize of 6.15 in the original studies and 1.37 in the replication studies) [41]. In our study, translation\nsuccess was flagged if the following conditions were satisfied, depending on the allowed level of shrinkage:\n1. ps < ˜α[shrink=0%] = 0.025 when allowing for no shrinkage;\n2. ps < ˜α[shrink=25%] = 0.0361 when allowing for moderate shrinkage; or,\n3. ps < ˜α[shrink=50%] = 0.0596 when allowing for high shrinkage.\nFinally, the controlled scepticalp-value p∗\ns, another recalibration presented in Micheloud, Balabdaoui,\nand Held [39], guarantees control of the overall T1E rate at levelα2. Translation success was flagged if\np∗\ns < 0.025.\nNote that whenever the animal finding indicated a harmful effect (i.e., the effect goes in the opposite\ndirection than what was expected), we implemented all scepticalp-value metrics in a way that forced\nthem to flag failure. This approach is valid, as no version of the scepticalp-value would ever flag success\nif the original or the animal finding has a very highp-value.\n2.8 Performance measures\nTo evaluate the performance of each translation success metric, we calculated and compared the propor-\ntion P of synthetic pairs of animal and human findings for which the metric flagged successful translation\nunder the different simulation conditions. The denominator for this proportion depends on the animal\nfinding (i.e., the results of the meta-analysis ofk animal studies) and the different continuation criteria\n(strict, lenient, none) described in Section 2.6. This leads to the following three versions of P:\n• Under the strict criterion: Pstrict = Number of pairs flagged successful\nNumber of statistically significant animal findings,\n• Under the lenient criterion: Plenient = Number of pairs flagged successful\nNumber of animal findings that found a beneficial effect,\n• Under no criterion: Pno = Number of pairs flagged successful\nNumber of simulation repetitions.\nUnder the assumption of animal and human null effects, this proportion reflects the overall T1E rate\n– the rate offalse positive translation success. The lower this proportion, the better a metric is at\nuncovering false translation failure. Under the assumption that both animal and human effects are\nnot null, the proportion can be interpreted as translation power, i.e., the probability oftrue positive\ntranslation success. The higher the proportion of true positive translation success, the better the metric\nis suited to “correctly” declare translation success under the chosen simulation conditions.\nWe used so-called “nested loop plots” to represent and compare the proportions for the different metrics\nacross simulation conditions as recommended by Rücker and Schwarzer [42]. The combinations of sim-\nulation conditions are ordered and arranged on the horizontal axis, while the proportion of successful\ntranslations is presented on the vertical axis1.\n1We refer to the caption of Figure 1.(a) for a brief description of how to interpret these plots.\n9\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n2.9 Monte Carlo uncertainty and number of simulation repetitions\nThe number of simulation repetitions was calculated based on a maximum desired Monte Carlo standard\nerror (MCSE) of 0.5% for P [43]. We considered the “worst-case” scenario of P= 0.5 (i.e., the metric\nis not better than tossing a coin), as well as the strictest criterion for a human study to be performed.\nFrom this, we obtained a maximum of400′000 animal studies to simulate in order to move on to at\nleast 10′000 human studies, while maintaining a maximum MCSE of0.5%. For simplicity, we simulated\n400′000 animal studies under all combinations of simulation conditions.\n2.10 Implementation\nOur simulation study was implemented inR (version 4.5) and designed using theSimDesign package\n[44]. We used theBFr function from theBayesRep package to compute the replication BF [45], and the\nReplicationSuccess package for all versions of the scepticalp-value [46]. Following Pawel et al. [47], we\nrecordedandreportedtheproportionofmissingness. Thisisacommonissueinsimulationstudiesinwhich\nproblems such as non-convergence of optimization algorithms may cause some simulation repetitions and\nconditions to yield invalid outputs, leading to missing values for the performance measures.\n3 Results\n3.1 Characteristics of simulated animal and human studies\nWe first illustrate the impact of our simulation design choices on the animal and human studies separately,\nand verify that the simulations were performed as expected. For this, Figure 1.(a) shows a nested loop\nplot with the proportion of significant animal and human findings (one-sidedp < α = 0.025) according\nto the simulation conditions.\nAs expected, both animal and human studies show a T1E rate (i.e., proportion under the null) of about\nα = 0.025 under the null hypothesis of no effect (i.e.,µA = 0 and µH = 0) combined with no heterogeneity\nacross studies. Increasing heterogeneity increases the T1E rates. For the animal findings, the T1E rate\ndecreases with an increasing number of pooled studiesk.\nThe human study sample size ofnH = 107 was chosen in order to achieve 80% power assuming a small\neffect size and no heterogeneity. Accordingly, under these conditions, we also find that about 80% of\nthe human findings are significant. Also as expected, the power decreases with increasing heterogeneity.\nSimulating under the large human effect using the same sample sizenH, naturally results in higher power\nclose to 1, except when heterogeneity is large.\nOn the other hand, the animal findings have low power when under the simulation condition of a small\nanimal effect. As expected, power increases with increasingk and the larger animal sample size, but still\nremains rather low. Increasing heterogeneity across the animal studies further lowers the proportion of\nsignificant animal findings. Finally, simulating under the large animal effect results in highly powered\nfindings, almost 100% power, except in the case of high heterogeneity.\nThen, Figure 1.(b) shows that conditioning the decision to conduct a human study on the animal finding\n(being beneficial or significant) results in overestimated effect sizes for the animal finding. The stricter\nthe decision criterion, the more inflated the estimated average effect size in the animal studies.\nMissingness Our simulation study was also affected by missingness, though only in rare cases. Specif-\nically, missingness occurred only in the data generating mechanism (see classification in Pawel et al.\n[47]) and was due to non-convergence of the Fisher scoring algorithm of the meta-analysis of the animal\nstudies using therma function. A table in our online supplement (https://rachelheyard.pages.uzh.\n10\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n(a)\n2x3x3x4 = 72 scenarios\n0\n20\n40\n60\n80\n100\nProportion of significant findings (%)Animal sample size (small, large)\nAnimal effect size (null, small, large)\nHeterogeneity across animal studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nAnimal findings\n3x3 = 9 scenarios\n0\n20\n40\n60\n80\n100\nα = 2.5 %\n1 − β = 80 %\nHuman effect size (null, small, large)\nHeterogeneity across human studies (none, low, high)\nHuman findings\n(b)\n2x3x3x4 = 72 scenarios\n−28\n−23\n−18\n−13\n−8\n−3\nAverage animal effect sizes\nAnimal sample size (small, large)\nAnimal effect size (null, small, large)\nHeterogeneity across animal studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nNo criterion\nLenient\nStrictµA = 0\nµA = −4.44\nµA = − 24.37\nFigure 1: (a) Nested loop plot of the proportion of statistically significant animal and human findings\nover all simulation repetitions depending on the simulation conditions, i.e. animal and human effect sizes,\nheterogeneity across animal and human studies, animal study sample sizes, and the number of animal\nstudies pooled together to obtain the animal finding. The dotted horizontal lines represent a proportion\nof 2.5% and 80%. The legend under each plot shows which of the progressively thinner columns in the\nplot correspond to which combination of simulation conditions. Each horizontal line segment contains\nthe proportion of significant findings under each combination of conditions. For example, the segment\nhighlighted with arrow represents the proportion of significant findings in the animal studies, when the\nsmaller sample size was used, the animal effect size was small, there was no heterogeneity across the\nanimal studies, and 5 animal studies were pooled.(b) Nested loop plot of the average animal effect size\nover all simulation repetitions, depending on the simulation conditions and the decision criterion applied\non the animal finding. The average effect size observed in the human studies is not affected by the applied\ncriterion. Note that since the criterion was not added as a simulation condition, the represented data is\ncorrelated, as the same simulation repetitions are used to calculate the average effect size for the strict,\nlenient and no criterion.\n11\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nch/translation_simulation/) summarizes the proportion of missingness for each combination of sim-\nulation condition, with a maximum of 0.0025% (i.e., 10 missing values out of 400’000 repetitions). These\nrepetitions were omitted in analyses.\n3.2 Performance of translation success metrics\nFigure 2 shows the proportion of animal-human pairs for which the different metrics flagged success-\nful translation across simulation conditions. The figure specifically shows the proportion Pno (no cri-\nterion) and the small animal sample size nA = 10. Note that Figure 2 does not include the re-\nsults for the replication BF and the meta-analysis for readability reasons as they behave very differ-\nently (see the corresponding Figure A.1 in the appendix). Our online supplement allows the reader\nto zoom into the different plots and also contains the results with the larger animal sample size; see\nhttps://rachelheyard.pages.uzh.ch/translation_simulation/.\nAssuming a large animal effect and a small human effect(bottom center plot in Figure 2) This\ncombination of animal and human effect sizes is closest to the results from the meta-analysis in Terstappen\net al. [26], and is therefore potentially the most realistic in the translation setting. Here, a well-performing\nmetric should find a relatively high proportion of translation successes, i.e., high translation power.\nWhen no heterogeneity is present in either animals or humans, the translation power for all metrics in\nthe figure is at least1 − β = 80%. The two-trials rule and the weighted Edgington are both equal to 80%\n(lines overlap). Unweighted Edgington behaves similarly with a sightly higher proportion. The controlled\nand golden (high shrinkage) scepticalp-values find the highest translation power.\nIncreasing the heterogeneity in human studies decreases the translation power of all metrics and brings\nthem closer together. When heterogeneity of the human studies is low and animal heterogeneity is\nnone, all metrics are close to(1 − β)2. These results barely change when increasing the animal study\nheterogeneity from none to low. This might be due to the fact that the relative sample sizec = nH/nA is\nalways larger than 1, even if the sample size of the animal finding is artificially increased with increasing\nk. A c > 1 forces some metrics to give more weight to the human study; therefore, even slight increases\nin the heterogeneity across human studies affects translation results. The translation power of the three\ngolden scepticalp-values generally increases withk, as a higherk leads to a higher chance of observing\na significant animal finding. The translation power of the three golden scepticalp-values is lowest when\nanimal study heterogeneity is high, which can be explained by the decrease in power of the individual\nanimal studies. The controlled sceptical p-value has a similar pattern with respect to k and animal\nheterogeneity, but increasingk to 5 countermeasures and its translation power is equal to 80%.\nThe metric based on a meta-analysis outperforms all other metrics in most conditions represented in\nFigure A.1. From previous research [19, 39], we know that if either the animal or the human finding is\nconvincing, the likelihood for the meta-analysis to flag success is high, regardless of the evidence in the\nother study. The replication BF results in very low proportions of successful translation when there is no\nor low heterogeneity across animal and human studies, because the effect sizes from animal and human\nfindings are too inconsistent. The results for the replication BF are more comparable to the results of\nthe other metrics in the presence of high study heterogeneity in either animals or humans.\nUnder the lenient criterion, the conclusions are the same. Under the strict criterion, most conclusions\nhold, while the proportions for the two-trials rule, both Edgington and the controlledp-value are now\nindependent from increases ink and increases in the level of heterogeneity across animal studies. Larger\nanimal sample size per group increases all proportions slightly, apart from the proportion for the repli-\ncation BF, which decreases.\n12\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAssuming null effects in animal and human studies (upper left corner in Figure 2) Here, a\nwell-performing metric should find a low proportion of translation successes, i.e., a low overall T1E rate\nor false positive translation success rate. All metrics except the replication BF in (Figure A.1) control\nthe overall T1E rate at α2 when there is no animal or human study heterogeneity. An increase in\nhuman study heterogeneity inflates the proportion of false positive translations. The two-trials rule is\nleast affected by changes in heterogeneity across human studies, followed by the unweighted Edgington,\nweighted Edgington and controlled scepticalp-value. The golden scepticalp-value (no shrinkage) keeps\nthe overall T1E rate low, especially when there is no heterogeneity across studies of either animals or\nhumans, which aligns with the theoretically expected pattern [38]. The golden scepticalp-values allowing\nfor moderate/high shrinkage permit weaker animal findings to translate even if shrinkage is observed in\nthe human study, but raise the overall T1E rate.\nMore animal study heterogeneity also increases the proportion of translation successes for all metrics.\nFor all golden scepticalp-values, the increase in the proportion withk when there is no animal study\nheterogeneity is due to the corresponding decrease in relative sample sizec. However, when there is low\nor high animal study heterogeneity, an increase ink leads to a decrease in the overall T1E rate for all\nmetrics in Figure 2. This is likely related to the fact that increases ink decrease the partial animal T1E\nrate when animal study heterogeneity is low or high (see Figure 1.(a)). Generally, when there is no or\nlow heterogeneity across animal studies combined with any level of heterogeneity across human studies,\nthe two-trials rule performs relatively well, with the overall T1E rate mostly belowα, and the weighted\nEdgington follows closely behind. However, when the heterogeneity across animal studies is high, the\ngolden scepticalp-value (no and moderate shrinkage) performs better compared to the other metrics.\nThe metric based on meta-analysis and the replication BF, visible in Figure A.1, results in very high\noverall T1E rates. The replication BF weights the evidence of the “replication”, i.e., the human study,\nmore heavily than the “original” animal finding. Increases in human study heterogeneity increase the\npartial T1E rate of the human finding; likewise, the same is true for the overall T1E rate in the case of\nthe replication BF. The meta-analysis metric treats the animal and human findings as interchangeable and\nit is therefore enough if just one of the findings is very convincing to flag success. Since high heterogeneity\nin human studies results in an increased risk of a false positive human result, the overall T1E rate of\nthe meta-analysis metric increases as well, up to 40% in extreme cases. Neither the replication BF nor\nthe meta-analysis metric are affected much by increases in animal study heterogeneity. An increase in\nk decreases the overall T1E for the meta-analysis slightly, while it increases the overall T1E for the\nreplication BF, especially when there is high heterogeneity across animal studies.\nResults under the lenient criterion can be studied in the online appendix and follow similar trends.\nUnder the strict criterion, translation success is conditional on the animal finding being significant.\nConsequently, the two-trials rule, both versions of Edgington’s method and the controlled scepticalp-\nvalues, which previously controlled the overall T1E rate atα2 when there was no study heterogeneity in\neither animals or humans, now do so at levelα = 0.025. The golden scepticalp-value (no shrinkage) now\nyields the lowest translation success rates across all scenarios. The results for the replication BF and the\nmeta-analysis are more comparable to those of the other metrics, aside from the replication BF when\nhuman study heterogeneity is high. Under the strict criterion, the overall T1E tends to increase withk,\nexcept when there is high heterogeneity across animal studies.\nAssuming small animal and human effects(center plot in Figure 2) Here, we observe translation\nsuccess rates that are much lower than what we would expect, i.e.,< (1 − β)2 = 0.64, except for the\nreplication BF in Figure A.1. This is due to the fact that the animal studies withnA = 10 have insufficient\npower to detect a small effect. As shown in Figure A.3, the results look slightly better with the larger\nanimal study sample size. In addition, by artificially increasing the sample size of the animal finding with\n13\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nk, we also observe that the rates for all metrics increase at least slightly. Notably, however, this trend is\nreversed when there is high heterogeneity across animal studies. Increases in heterogeneity across studies\nof any species decreases the translation power for all metrics except the replication BF and meta-analysis.\nThe replication BF results in the highest proportion of successful translations under all conditions, gen-\nerally followed by the meta-analysis. This can be explained by the fact that the replication BF puts more\nweight on the human study, and meta-analysis treats human and animal findings as interchangeable.\nAmong the golden scepticalp-values, the version allowing for no shrinkage would be the most appro-\npriate here, since the true animal and human effect sizes are assumed to be equal. This metric leads\nto the lowest translation power, apart from when the heterogeneity across human studies is high; then,\nthe two-trials rule leads to similar or smaller translation success rates. The controlled scepticalp-value\ngenerally yields higher translation power compared to the other metrics, and is even the highest when\nthere is high animal study heterogeneity, aside from the replication BF and meta-analysis. The golden\nsceptical p-value (high shrinkage) also performs similarly well, especially with no or low animal study\nheterogeneity.\nUnder the strict criterion (see online appendix), the two-trials rule, unweighted and weighted Edgington\nand the controlled sceptical p-value are approximately equivalent to the power of the human studies\n(1 − β = 0.8), as illustrated in Figure 1.(a). The golden scepticalp-value (no shrinkage) with borderline\nsignificant results generally finds the lowest proportion of translation success across conditions. The\nlow-powered animal studies might lead to overestimated effect sizes for the animal findings, which is\nmost penalised by this version of the golden scepticalp-value (no shrinkage). In addition, weighted\nEdgington finds proportions of translation success that are equivalent or slightly smaller than those of\nunweighted Edgington, while the proportions are larger when applying no criterion. This is because\nweighted Edgington puts less weight on the animal findings, which are heavily inflated under the strict\ncriterion.\nAssuming large animal and human effects (bottom right plot in Figure 2) When applying no\ncriterion, most metrics find a translation power of almost 100% under most simulation conditions, except\nwhen there is high heterogeneity across animal studies, as in that case the animal studies have low power.\nTranslation success rates are even closer to 100% under the strict criterion.\nAssuming a small animal effect and a human null effect(center left plot in Figure 2) Under\nthis combination of effect sizes, translation success should not occur. Indeed, the translation success\nrates for all metrics are generally small (< α) when there is no heterogeneity across animal or human\nstudies. However, the proportion increases substantially with increasing levels of heterogeneity across\nhuman studies, and decreases with increasing levels of heterogeneity across animal studies. The meta-\nanalysis, replication BF, scepticalp-value (high shrinkage) and controlled scepticalp-value lead to the\nhighest proportions, while the golden scepticalp-value (no shrinkage) and the two-trials rule lead to the\nlowest proportions. Note that the animal studies have low power to detect a small effect.\nAssuming an animal null effect and a small human effect(top center plot in Figure 2) When\nthe human effect is small, for which the human studies were powered at 80%, and the animal effect is\nnull, all metrics but the replication BF and the meta-analysis generally result in translation success rates\nclose to α. The proportion for the replication BF is close to the power of the human studies. Under the\nstrict criterion, all metrics get closer to the human study power. Interestingly, under the strict criterion,\nthe metric based on the meta-analysis is one of the metrics with the lowest proportions. This might be\ndue to the human studies being powered at 80% and not higher.\n14\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n5\n10\nα2\nα\n5\n10\n15\n20\n25\n30\n35\nα2\nα\n0\n5\n10\n15\n20\n25\n30\n35\n40\n45\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα\n0\n5\n10\n15\nα\nTwo−trials rule\nEdgington (unweighted)\nEdgington (weighted)\nGolden (no shrinkage)\nGolden (moderate shrinkage)\nGolden (high shrinkage)\nControlled\n0\n10\n20\n30\n40\n50\n60\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\n0\n5\n10\n15\n20\n25\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\nµH = 0 µH = −4.44 µH = −24.37\nµA = 0µA = −4.44µA = −24.37\nProportion (%)\n3 × 3 × 4 = 36 scenarios\nFigure\n2: Grid of nested loop plots of the proportions of animal-human pairs for which the different metrics flagged successful translation across simulation\nconditions under no criterion. Each of the plots in the grid represent another animal-human finding combination. In the first column, for example, the human\nstudies are all simulated under the null hypothesis of no effect. Note that the results for the replication BF and the meta-analysis are not shown here for better\nreadability. The dotted horizontal lines representα2 = 0.000625, α = 0.025, 1 − β = 0.8 and (1 − β)2 = 0.64 . All animal studies in this representation are\nsimulated with a small sample size per group (nA = 10).\n15\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAssuming a large animal effect and a human null effect(bottom left plot in Figure 2) Here,\nthe animal studies have high power, though power decreases with increasing heterogeneity across animal\nstudies. Accordingly, when animal study heterogeneity is non-existent or low, there is a high chance\nof a very convincing animal finding, which then results in high translation success rates for the meta-\nanalysis metric. The replication BF tends to be the most conservative unless there is high animal study\nheterogeneity. The remaining metrics all perform similarly well, are only slightly affected by changes in\nk and generally follow the partial T1E rate of the human studies.\nAssuming a small animal effect and a large human effect(center right plot in Figure 2) Here,\nthe human studies are highly powered and the animal findings have low power. Hence, applying the strict\ncriterion (see online appendix) leads to a translation success rate of almost 100% for all conditions except\nwhen heterogeneity across human studies is high, where it is then close to 90%. When no criterion is\napplied, the high-powered human studies still lead to a proportion of 1 or close to 1 for the replication\nBF and the meta-analysis. The remaining metrics are more conservative, with the two-trials rule yielding\nthe smallest proportions across conditions. Proportions for all metrics increase with increasingk unless\nthere is high animal study heterogeneity, and decrease with increasing human study heterogeneity.\nAssuming an animal null effect and a large human effect(top right plot in Figure 2) Here, all\nmetrics aside from the replication BF and the meta-analysis behave as one would expect: they rarely\nflag translation success. When there is no heterogeneity across animal studies, the proportion ranges\nfrom α for the two-trials rule to 10% for the golden scepticalp-value (high shrinkage) and the controlled\nsceptical p-value. These proportions further increase with increasing levels of animal study heterogeneity.\nApplying the strict criterion again results in proportions close to 1 for all metrics.\n4 Discussion\nIn our simulation study, we investigated whether metrics used or developed to assess replication success\ncan be applied and are useful in the context of translation of results from animal studies to human\nstudies. Our study was motivated by recorded cases of translation failure in biomedical research. We\naimed to assess how well various statistical metrics capture the concept of translation under a wide\nrange of simulated conditions, including differences in effect sizes, effect size heterogeneity, animal study\nsample sizes and the number of animal studies pooled together. For this, we simulated animal and human\nstudies using parameters informed by a real-world meta-analysis of prenatal amino acid supplementation\non maternal blood pressure [26]. We also simulated different scenarios for the decision to move on to\na human study: (1) any animal finding leads to a subsequent human study, (2) only beneficial animal\nfindings lead to a subsequent human study, and (3) only significant beneficial animal findings lead to a\nsubsequent human study. Based on the pairs of findings from the simulated animal and human studies,\nwe evaluated nine metrics that have previously been discussed in the replication literature.\nWe show that the performance of the different metrics highly depends on the simulation conditions.\nFirst, when both animal and human true effects are null, most metrics, except for the replication BF\nand meta-analysis, control the overall T1E rate close to the theoreticalα2 under no heterogeneity. When\nheterogeneity increases, especially between human studies, the overall T1E increases. When both animals\nand humans had non-null effects, translation power was most influenced by whichever of the animal or\nhuman finding that had lower power. For example, under the conditions of small effects in both animals\nand humans and small animal sample sizes, translation power fell below(1 − β)2. Conversely, assuming\nlarge effects in both animals and humans yielded near-perfect translation power except in cases of high\nheterogeneity across animal studies, in which case translation power was lower.\n16\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAsymmetric effect size scenarios revealed systematic tendencies. Meta-analysis generally flagged success\nmore often, driven by strong evidence in either animals or humans, while the two-trials rule and the\ngolden sceptical p-value (no shrinkage) were more conservative and aligned more closely with the weaker\nof the two findings. Replication BF did not perform well whenever asymmetric effect sizes were simulated.\nIncreasing the number of animal studies that were pooled (k) typically improved translation power when\nanimal effects were non-null and heterogeneity was low, but had little benefit and even a negative impact\nwhen animal study heterogeneity was high, which is however often the case in practice.\nConditioning on significant animal findings as a strict criterion to move on to human testing, inflated\nanimal effect size estimates and affected the operating characteristics of the metrics, e.g., sometimes\nsubstantially increasing T1E rates or power. Overall, no single metric was uniformly optimal. The\ncontrolled sceptical p-values and weighted Edgington performed relatively well across many scenarios,\nwhile replication BF and meta-analysis were highly sensitive to strong findings in either animals or\nhumans. Golden sceptical p-values offered more conservative control at the cost of reduced power when\ntrue effects were small.\nA conceptual challenge uncovered in our simulation study was how to interpret cases in which the true\neffect sizes in animal and human differ in complex ways. For example, is a “translation success” desirable\nin the case where the true animal effect is null but the human effect is small? Most probably it is not.\nSuch cases would benefit from a deeper discussion in the community of what constitutes a successful\ntranslation, especially because animal testing is often treated as a precursor to human studies rather\nthan an end in itself. It is therefore important to recognize that translation differs fundamentally from\nreplication, because in the translation setting the human finding is the reference point and the target\npopulation against which success is ultimately judged.\nLimitations\nThis study has various limitations. Our simulation study assumes a degree of comparability of effect sizes\nbetween animal and human studies that may not exist in practice. In reality, the magnitude of effects\noften differ substantially across species due to biological, methodological and environmental factors. The\ntype of effects and outcome measurements investigated in animals might differ from those that are of\ninterest in human studies. Human studies typically progress through various clinical phases with distinct\ngoals and our simulation study did not distinguish between these phases. Our choice to pool a maximum\nof five animal studies to form a single “animal finding” may be overly simplistic. In real-world settings,\nthe decision to move to testing in humans is often not (solely) based on the statistical significance\nand direction of effect in a (relatively small) set of animal studies. A broader array of factors may be\nconsidered, including pharmacokinetics, safety profiles and ethical considerations. Our study is merely\nlooking at the statistical aspect of translation, which is only one component of a more complex decision-\nmaking process. While the choice of our study parameters was informed by a real meta-analysis, our\nsimulations are based on a single dataset and domain, which might limit the generalisability of our results.\nOther biomedical fields might exhibit different patterns of effect size differences and heterogeneity. To\nallow for the broadest possible range of scenarios, we used a fully factorial design and assigned all possible\neffect sizes and levels of heterogeneity to both animal and human studies. This may have introduced\nunrealistic scenarios. Finally, we focussed on a specific set of metrics. Other, more appropriate metrics\nmight exist that we are unaware of.\nRecommendations\nOur findings highlight that the choice of translation success metrics, along with the design features of\nboth animal and human studies, can meaningfully influence conclusions about “translatability”. The\n17\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nlow translation power of small-sample animal studies, even if effects are truly present, suggests that\npooling multiple studies or increasing samples sizes is crucial to reduce false negatives and avoid inflated\neffect size estimates, especially when results will be used to justify human clinical trials. Special attention\nshould also be given to heterogeneity when interpreting translation failures, as even modest heterogeneity\nacross studies can reduce the chance of translation success according to most metrics. Our results also\nsuggest caution when basing the justification of clinical trials solely on statistical significance in animal\nfindings, i.e., strict criterion, as this can lead to overly optimistic expectations for human outcomes.\nWhen assessing translation outcomes, metrics that balance information from both animals and humans –\nsuch as controlled scepticalp-values or weighted Edgington – may provide more robust conclusions than\nmetrics that are driven by strong evidence in just one species (e.g., replication BF focusing on mainly\nhuman findings, and meta-analysis).\nConclusions\nWe conclude that metrics developed for assessing replication success can offer valuable insights for assess-\ning translation success. However, their utility depends strongly on the context, underlying assumptions,\nandthecharacteristicsoftheavailableevidence. Nosinglemetricperformedoptimallyacrossallsimulated\nscenarios. A combined approach, using multiple metrics alongside an understanding of their respective\nstrengths and limitations, is recommended to assess when and how animal findings translate to human\noutcomes. Future research is needed to explore and better understand the behaviour of the metrics in\nthe translation setting from a theoretical perspective to draw generalisable conclusions in biomedical\ncontexts.\nData and software availability\nAll data and code file to reproduce our simulation results, this manuscript and the online supple-\nment are available via GitLab,https://gitlab.uzh.ch/rachelheyard/translation_simulation. A\ncitable snapshot of the repository at the time writing is archived athttps://doi.org/10.5281/zenodo.\n13587432.\nAcknowledgements\nWethankGillianCurrieandBernhardVoelklforvaluablefeedbackonanearlierversionofourmanuscript.\nAdditionally, we would like to thank the iRISE consortium, and specially work package 1, for continuous\nfeedback in the conceptualization and reporting of our work.\nFunding statement\nRH and KW receive funding from iRISE. iRISE receives funding from the European Union’s Horizon\nEurope research and innovation programme under grant agreement No 101094853. Views and opinions\nexpressed are however those of the author(s) only and do not necessarily reflect those of the European\nUnion or the European Research Executive Agency (ERA). Neither the European Union nor the ERA can\nbe held responsible for them. iRISE also receives funding from the Swiss State Secretariat for Education,\nResearch and Innovation (SERI): Direct Funding for Collaborative Projects as part of the transitional\nmeasures, and from UK Research and Innovation (UKRI). BVI receives funding from the Swiss National\nScience Foundation under grant number 407940_206504.\n18\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAuthor contributions\n• Conceptualization: CJH, SP, KEW, BVI, RH\n• Data curation: CJH, KEW\n• Formal Analysis:CJH, RH\n• Funding acquisition:KEW, BVI, RH\n• Methodology: CJH, SP, RH\n• Project administration: RH\n• Software: CJH, RH\n• Supervision: RH\n• Visualization: CJH, RH\n• Writing – original draft:CJH, RH\n• Writing – review & editing:CJH, RH SP, KEW, BVI\nReferences\n[1] B. V. Ineichen, E. Furrer, S. L. Grüninger, W. E. Zürrer, and M. R. Macleod. “Analysis of animal-to-\nhuman translation shows that only 5% of animal-tested therapeutic interventions obtain regulatory\napproval for human applications”. In:PLoS Biology22.6 (2024), e3002667.\n[2] C. H. Leenaars et al. “Animal to human translation: A systematic scoping review of reported\nconcordance rates”. In:Journal of translational medicine17 (2019), pp. 1–22.\n[3] D. G. Hackam and D. A. Redelmeier. “Translation of research evidence from animals to humans”.\nIn: Jama 296.14 (2006), pp. 1727–1732.\n[4] S. Perrin. “Preclinical research: Make mouse studies work”. In:Nature 507.7493 (2014), pp. 423–425.\n[5] B. Voelkl et al. iRISE Reproducibility Glossary. 2024. doi: 10.17605/OSF.IO/BR9SP.\n[6] A. Schmidt-Pogoda et al. “Why most acute stroke studies are positive in animals but not in patients:\na systematic comparison of preclinical, early phase, and phase 3 clinical trials of neuroprotective\nagents”. In:Annals of neurology87.1 (2020), pp. 40–51.\n[7] E.Wilsonetal.“Designing,conducting,andreportingreproducibleanimalexperiments”.In: Journal\nof Endocrinology258.1 (2023).\n[8] S. C. Landis et al. “A call for transparent reporting to optimize the predictive value of preclinical\nresearch”. In:Nature 490.7419 (2012), pp. 187–191.\n[9] J. D. Wallach, K. W. Boyack, and J. P. A. Ioannidis. “Reproducible research practices, transparency,\nand open access data in the biomedical literature, 2015–2017”. In:PLOS Biology16.11 (2018). Ed.\nby U. Dirnagl, e2006930.doi: 10.1371/journal.pbio.2006930.\n[10] K. S. Button et al. “Power failure: why small sample size undermines the reliability of neuroscience”.\nIn: Nature Reviews Neuroscience14.5 (2013), 365–376.doi: 10.1038/nrn3475.\n[11] A. Bespalov et al. “Failed trials for central nervous system disorders do not necessarily invalidate\npreclinical models and drug targets”. In:Nature Reviews Drug Discovery15.7 (2016), pp. 516–516.\n19\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n[12] R.-D. Gosselin. “Insufficient transparency of statistical reporting in preclinical research: a scoping\nreview”. In:Scientific Reports11.1 (2021). doi: 10.1038/s41598-021-83006-5.\n[13] C. Kilkenny et al. “Survey of the quality of experimental design, statistical analysis and reporting\nof research using animals”. In:PLoS ONE4.11 (2009), e7824.\n[14] E. S. Sena, H. B. Van Der Worp, P. M. Bath, D. W. Howells, and M. R. Macleod. “Publication bias\nin reports of animal stroke studies leads to major overstatement of efficacy”. In:PLoS Biology8.3\n(2010), e1000344.\n[15] D. Fanelli. “Negative results are disappearing from most disciplines and countries”. In:Scientomet-\nrics 90.3 (2012), pp. 891–904.\n[16] P. Flecknell. “Replacement, reduction, refinement”. In:ALTEX-Alternatives to animal experimen-\ntation 19.2 (2002), pp. 73–78.\n[17] National Academies of Sciences, Engineering, and Medicine. Reproducibility and Replicability in\nScience. National Academies Press, 2019.doi: 10.17226/25303.\n[18] R. Heyard et al. “A scoping review on metrics to quantify reproducibility: a multitude of questions\nleads to a multitude of metrics”. In:Royal Society Open Science12.7 (2025). doi: 10.1098/rsos.\n242076.\n[19] F. Freuli, L. Held, and R. Heyard. “Replication success under questionable research practices—a\nsimulation study”. In:Statistical Science38.4 (2023), pp. 621–639.\n[20] J. Muradchanian, R. Hoekstra, H. Kiers, and D. van Ravenzwaaij. “How best to quantify replication\nsuccess? A simulation study on the comparison of replication success metrics”. In:Royal Society\nOpen Science8.5 (2021), p. 201697.\n[21] J. Muradchanian, R. Hoekstra, H. Kiers, and D. van Ravenzwaaij. “Evaluating meta-analysis as\na replication success measure”. In:PLoS ONE 19.12 (2024). Ed. by D. Purić, e0308495.doi: 10.\n1371/journal.pone.0308495.\n[22] L. Held. “Beyond the two-trials rule”. In:Statistics in Medicine(2024).\n[23] B. A. Nosek and T. M. Errington. “Making sense of replications”. In:eLife 6 (2017).doi: 10.7554/\nelife.23383.\n[24] B. S. Siepe et al. “Simulation studies for methodological research in psychology: A standardized\ntemplate for planning, preregistration, and reporting.” In:Psychological Methods(2024). doi: 10.\n1037/met0000695.\n[25] C. J. Huang and R. Heyard. A simulation study to quantify successful translation of results from\npreclinical studies to human trials. 2024. doi: 10.17605/OSF.IO/BZXVY.\n[26] F. Terstappen et al. “Prenatal amino acid supplementation to improve fetal growth: a systematic\nreview and meta-analysis”. In:Nutrients 12.9 (2020), p. 2535.\n[27] C. R. Hooijmans et al. “Remyelination promoting therapies in multiple sclerosis animal models:\na systematic review and meta-analysis”. In:Scientific Reports9.1 (2019). doi: 10.1038/s41598-\n018-35734-4.\n[28] J. Y. Chien, S. Friedrich, M. A. Heathman, D. P. de Alwis, and V. Sinha. “Pharmacokinet-\nics/pharmacodynamics and the stages of drug development: role of modeling and simulation”. In:\nThe AAPS Journal7 (2005), E544–E559.\n[29] P. C. Lind et al. “Translation from animal studies of novel pharmacological therapies to clinical\ntrials in cardiac arrest: A systematic review”. In:Resuscitation 158 (2021), pp. 258–269.\n20\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n[30] G. K. Rosenkranz. “A Generalization of the Two Trials Paradigm”. In:Therapeutic Innovation &\nRegulatory Science57.2 (2022), 316–320.doi: 10.1007/s43441-022-00471-4.\n[31] R. D. Cousins. Annotated Bibliography of Some Papers on Combining Significances or p-values.\n2007. doi: 10.48550/ARXIV.0705.2209.\n[32] C. Röver and T. Friede. “Investigating the Heterogeneity of “Study Twins””. In:Biometrical Journal\n66.6 (2024). doi: 10.1002/bimj.202300387.\n[33] J. Verhagen and E.-J. Wagenmakers. “Bayesian tests to quantify the result of a replication attempt.”\nIn: Journal of Experimental Psychology: General143.4 (2014), p. 1457.\n[34] H. Jeffreys. The theory of probability. 3rd ed. Oxford Classic Texts in the Physical Sciences. London,\nEngland: Oxford University Press, 1998.\n[35] E. S. Edgington. “An additive method for combining probability values from independent experi-\nments”. In:The Journal of Psychology80.2 (1972), pp. 351–363.\n[36] L. Held, S. Pawel, and C. Micheloud. “The assessment of replicability using the sum of p-values”.\nIn: Royal Society Open Science(2024). doi: 10.48550/arXiv.2401.13615.\n[37] L. Held. “A new standard for the analysis and design of replication studies”. In:Journal of the\nRoyal Statistical Society Series A: Statistics in Society183.2 (2020), pp. 431–448.\n[38] L. Held, C. Micheloud, and S. Pawel. “The assessment of replication success based on relative effect\nsize”. In:The Annals of Applied Statistics16.2 (2022), pp. 706–720.\n[39] C. Micheloud, F. Balabdaoui, and L. Held. “Assessing replicability with the sceptical p-value: Type-I\nerror control and sample size planning”. In:Statistica Neerlandica77.4 (2023), pp. 573–591.\n[40] Open Science Collaboration. “Estimating the reproducibility of psychological science”. In:Science\n349.6251 (2015). doi: 10.1126/science.aac4716.\n[41] T. M. Errington et al. “Investigating the replicability of preclinical cancer biology”. In:eLife 10\n(2021). doi: 10.7554/elife.71601.\n[42] G. Rücker and G. Schwarzer. “Presenting simulation results in a nested loop plot”. In:BMC Medical\nResearch Methodology14.1 (2014). doi: 10.1186/1471-2288-14-129.\n[43] T. P. Morris, I. R. White, and M. J. Crowther. “Using simulation studies to evaluate statistical\nmethods”. In:Statistics in Medicine38.11 (2019), 2074–2102.doi: 10.1002/sim.8086.\n[44] R. P. Chalmers and M. C. Adkins. “Writing effective and reliable Monte Carlo simulations with the\nSimDesign package”. In:The Quantitative Methods for Psychology16.4 (2020), pp. 248–280.doi:\n10.20982/tqmp.16.4.p248.\n[45] S. Pawel and L. Held. The sceptical Bayes factor for the assessment of replication success. 2022.\ndoi: 10.1111/rssb.12491.\n[46] L. Held, C. Micheloud, S. Pawel, F. Gerber, and F. Hofmann.Design and Analysis of Replication\nStudies with ReplicationSuccess. 2022.doi: 10.32614/CRAN.package.ReplicationSuccess.\n[47] S. Pawel, F. Bartoš, B. S. Siepe, and A. Lohmann. “Handling Missingness, Failures, and Non-\nConvergence in Simulation Studies: A Review of Current Practices and Recommendations”. In:The\nAmerican Statistician(2025), pp. 1–18.doi: 10.1080/00031305.2025.2540002.\n21\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\nAppendix\nCompleteFigures -FigureA.1showstheresultsofthesimulationstudyforallmetrics, alsoreplication\nBF and meta-analysis, across scenarios when the animal studies’ sample size is fixed to 10 per group.\nFigure A.2 shows the same type of results for when the animal studies’ sample size is fixed to 20 per group,\nwhile Figure A.3 shows the zoomed-in results where replication BF and meta-analysis were dropped for\nreadability.\n22\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n10\n20\n30\n40\n50\n60\n70\n80\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nα2 α\n(1 − β)2\nTwo−trials rule\nReplication BF\nEdgington (unweighted)\nEdgington (weighted)\nGolden (no shrinkage)\nGolden (moderate shrinkage)\nGolden (high shrinkage)\nControlled\nMeta−analysis\n0\n10\n20\n30\n40\n50\n60\n70\n80\nα\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\nµH = 0 µH = −4.44 µH = −24.37\nµA = 0µA = −4.44µA = −24.37\nProportion (%)\n3 × 3 × 4 = 36 scenarios\nFigure\nA.1: Grid of nested loop plots of the proportions of animal-human pairs for which the different metrics flagged successful translation across simulation\nconditions under no criterion. Each of the plots in the grid represent another animal-human finding combination. In the first column, for example, the human\nstudies are all simulated under the null hypothesis of no effect. The dotted horizontal lines representα2 = 0.000625, α = 0.025, 1 − β = 0.8 and (1 − β)2 = 0.64 .\nAll animal studies in this representation are simulated with a small sample size per group (nA = 10).\n23\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n10\n20\n30\n40\n50\n60\n70\n80\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα2 α\n(1 − β)2\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nα2 α\n(1 − β)2\nTwo−trials rule\nReplication BF\nEdgington (unweighted)\nEdgington (weighted)\nGolden (no shrinkage)\nGolden (moderate shrinkage)\nGolden (high shrinkage)\nControlled\nMeta−analysis\n0\n10\n20\n30\n40\n50\n60\n70\n80\nα\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\nµH = 0 µH = −4.44 µH = −24.37\nµA = 0µA = −4.44µA = −24.37\nProportion (%)\n3 × 3 × 4 = 36 scenarios\nFigure\nA.2: Grid of nested loop plots of the proportions of animal-human pairs for which the different metrics flagged successful translation across simulation\nconditions under no criterion. Each of the plots in the grid represent another animal-human finding combination. In the first column, for example, the human\nstudies are all simulated under the null hypothesis of no effect. The dotted horizontal lines representα2 = 0.000625, α = 0.025, 1 − β = 0.8 and (1 − β)2 = 0.64 .\nAll animal studies in this representation are simulated with a larger sample size per group (nA = 20).\n24\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint \n\n5\n10\nα2\nα\n5\n10\n15\n20\n25\n30\n35\n40\n45\nα2\nα\n0\n5\n10\n15\n20\n25\n30\n35\n40\n45\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\nα\n0\n5\n10\n15\nα\nTwo−trials rule\nEdgington (unweighted)\nEdgington (weighted)\nGolden (no shrinkage)\nGolden (moderate shrinkage)\nGolden (high shrinkage)\nControlled\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\n0\n5\n10\n15\n20\n25\nα\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\n(1 − β)2\n0\n10\n20\n30\n40\n50\n60\n70\n80\n90\n100\nHeterogeneity across animal studies (none, low, high)\nHeterogeneity across human studies (none, low, high)\nNumber pooled animal studies (2, 3, 4, 5)\n(1 − β)2\nµH = 0 µH = −4.44 µH = −24.37\nµA = 0µA = −4.44µA = −24.37\nProportion (%)\n3 × 3 × 4 = 36 scenarios\nFigure\nA.3: Grid of nested loop plots of the proportions of animal-human pairs for which the different metrics flagged successful translation across simulation\nconditions under no criterion. Each of the plots in the grid represent another animal-human finding combination. In the first column, for example, the human\nstudies are all simulated under the null hypothesis of no effect. Note that the results for the replication BF and the meta-analysis are not shown here for better\nreadability. The dotted horizontal lines representα2 = 0.000625, α = 0.025, 1 − β = 0.8 and (1 − β)2 = 0.64 . All animal studies in this representation are\nsimulated with a larger sample size per group (nA = 20).\n25\n . CC-BY 4.0 International licenseIt is made available under a \nperpetuity. \n is the author/funder, who has granted medRxiv a license to display the preprint in(which was not certified by peer review)preprint \nThe copyright holder for thisthis version posted November 9, 2025. ; https://doi.org/10.1101/2025.11.07.25339757doi: medRxiv preprint","source_license":"CC-BY-4.0","license_restricted":false}