Simulation results on the performance of statistical methods in cumulative meta analysis

Cumulative meta-analysis (CMA) is a process of updating the results of existing meta-analysis to incorporate new study results. This is a popular way to present time-varying evidence. We investigate the properties of CMA, suggest possible improvements and provide the first in-depth simulation study of the use of CMA and CUSUM methods for detection of temporal trends in random-effects meta-analysis. We use the standardized mean difference (SMD) as an effect measure of interest. For CMA, we compare the standard inverse-variance-weighted estimation of the overall effect using REML-estimated between-study variance $\tau^2$ with the sample-size-weighted estimation of the effect combined with Kulinskaya-Dollinger-Bjørkestøl (2011) (KDB) estimation of $\tau^2$. For all methods, we consider type 1 error under no shift and power under shift in the mean. To ameliorate the lack of power in CMA, we introduce the two-stage CMA, where the heterogeneity variance $\tau^2$ is estimated at stage 1 (first 5-10 studies), and the further CMA monitors a target value of effect, keeping the $\tau^2$ value fixed. We recommend the use of this two-stage CMA combined with cumulative testing for positive shift in $\tau^2$.