Statcheck in Canada: What Proportion of CPA Journal Articles Contain Errors in the Reporting of p-Values?

Using a computer program called “Statcheck,” a 2016 digital survey of several prestigious American and European psychology journals showed that the p-values reported in research articles failed to agree with the corresponding test statistics (e.g., F, t, χ2) at surprisingly high rates: nearly half of all articles contained at least one such error, as did about 10% of all null hypothesis significance tests. We investigated whether this problem was present in Canadian psychology journals and, if so, at what frequency. We discovered similar rates of p-value errors in Canadian journals over the past 30 years. However, we also noticed, a large number of typographical errors in the electronic versions of the articles. When we hand corrected a sample of our articles, the per-article error rate remained about the same, but the per test rate of errors dropped to 6.3%. We recommend that, in future, journals include explicit checks of statistics in their editorial processes.