Trial number and sample size do not affect accuracy of the I2- point estimate for testing selection bias risk in meta-analyses. A simulation study (Preprint)

Abstract Objectives: To establish the test sensitivity and specificity of the I2-point estimate for testing selection bias in meta-analyses under condition of large versus small trial sample size and large versus small trial number in meta-analyses and to test the null-hypotheses that the differences are not statistically significant. Methods: Simulation trials were generated in MS Excel, each consisting of a sequence of subject ID (accession) numbers representing trial subjects; a random sequence of allocation to group A or B and a random sequence of a simulated baseline variable (‘age’) per subject, ranging between 50 to 55. These simulation trials were included into five types of meta-analyses with large/small numbers of trials, as well as trials with large and small sample sizes. Half of the meta-analyses were artificially biased. All meta-analyses were tested using the I2 point estimate. The numbers of true positive, false positive, false negative and true negative test results were established. From these, the test sensitivity and specificity were computed for each of the meta-analysis types and compared. Results: All non-biased meta-analyses yielded true negative and all biased meta-analyses true positive test results, regardless of trial number and trial sample size. No false positive or false negative test results were observed. Accordingly, test sensitivities and specificities of 100% for all meta-analysis types were established and thus both null-hypotheses failed to be rejected. Conclusion: The results suggest that trial number and sample size in a baseline variable meta-analysis do not affect the test accuracy of the I2-point estimate.