Uncovering Null Effects in Null Fields: The Case of Homeopathy

Sigurdson, Sainani, and Ioannidis (Journal of Clinical Epidemiology, 2023) discussed homeopathy as a prototypical example of a “null field” where true effects are nonexistent and positive effect sizes reflect bias only. Based on a sample of published randomized placebo-controlled trials, they observed a remarkable effect in favor of homeopathy (Hedges’ g = 0.36). The authors concluded that this reflects “the average impact of the bias present in the field”. We argue that the estimated amount of bias largely depends on the meta-analytic measure used to quantify treatment effects. By applying a bias-corrected measure instead, we show that the maximum-likelihood estimate of the true effect reduces to virtually zero when selective publishing of significant results is appropriately taken into account. We conclude that inclusion of bias-corrected measures should become routine practice in meta-analyses. In line with Sigurdson, Sainani, and Ioannidis’ reasoning, we recommend “null fields” as domains to validate such measures.