Deep learning for robust meta-analytic estimation

Meta-analysis represents the promise of cumulative science--that each successive study brings us greater understanding of a given phenomenon. As such, meta-analyses are highly influential and gaining in popularity. However, there are well-known threats to the validity of meta-analytic results, such as processes like publication bias and questionable research practices which can cause researchers to massively overestimate the evidence in support of a claim. There are many statistical methods to correct for such bias, but no single method has been found to be robust in all realistic conditions. Here, I describe a method that merges statistical simulation and deep learning to achieve an unprecedented level of robust meta-analytic estimation in the face of numerous forms of bias and other historically problematic conditions. Furthermore, the resulting estimator, called DeepMA, has the unique property that it can easily evolve: As new conditions for which robustness is needed are identified, DeepMA can be re-trained to maintain high performance. Given the weaknesses that have been identified for meta-analysis, the current consensus is that it should serve as simply another data point, rather than residing at the top of the hierarchy of evidence. The novel approach I describe, however, holds the potential to eliminate these weaknesses, possibly solidifying meta-analysis as the platinum standard in scientific debate.