An Average Power Primer: Clarifying Misconceptions about Average Power and Replicability

The replication crisis heightened interest in methods for assessing the credibility of published research. One approach to evaluate published results is to estimate the average power of original studies based on observed data. Recent criticisms, however, have challenged the validity and usefulness of this approach, arguing that it involves a fundamental "ontological error," fails to predict replication outcomes accurately, and yields imprecise estimates. This article aims to address these critics. We clarify that using observed data to estimate true power is a standard inferential practice and does not constitute an ontological error. We argue that the primary purpose of average power estimation is not to predict the outcome of future replication studies, but the hypothetical outcome if original researchers had to replicate their studies with new samples. Lastly, we demonstrate that even when uncertainty is substantial, average power estimates provide valuable diagnostic information about the credibility of literatures, especially when selection for significance is present. An applied example using a Z-curve analysis of terror management research shows that seemingly strong evidence of over 800 significant results does not rule out the possibility that all results are false positives. We conclude that, despite limitations, average power estimation remains a valid and useful tool for evaluating the evidential value of published research.