Cohen’s Convention, the Seriousness of Errors, and the Body of Knowledge in Behavioral Science

An often-cited convention for discovery-oriented behavioral science research states that the general relative seriousness of the antecedently accepted false positive error rate of α = .05 be mirrored by a false negative error rate of β = .20. In 1965, Jacob Cohen proposed this convention to decrease a β-error typically in vast excess of .20. Thereby, we argue, Cohen (unintentionally) contributed to the wide acceptance of strongly uneven error rates in behavioral science. Although Cohen’s convention can appear epistemically reasonable for an individual researcher, the comparatively low probability that published effect size estimates are replicable renders his convention unreasonable for an entire scientific field. Appreciating Cohen’s convention helps to understand why even error rates (α = β) are “non-conventional” in behavioral science today, and why Cohen’s explanatory reason for β= .20—that resource restrictions keep from collecting larger samples—can easily be mistaken for the justificatory reason it is not. β = 0.20. Cohen’s convention not only ignores contexts of hypothesis testing where the more serious error is the β-error. Cohen’s convention also implies for discovery-oriented hypothesis testing research that a statistically significant observed effect is four times more probable to be a mistaken discovery than for a statistically significant true observed effect to be independently replicable. In the long run, Cohen’s convention thus is epistemically harmful to the development of a progressive science of human behavior, making its acceptance crucial in explaining the replication crisis in behavioral science. The balance between α- and β-errors generally ought to be struck using both epistemic and practical considerations. Yet epistemic considerations alone imply that making a genuine contribution to the body of knowledge in behavioral science requires error rates that are not only small but also symmetrical.