Research on “the Testing Effect” Routinely Conflates Direct and Forward Testing Effects: A Meta-Analysis of Testing Effects with Free Recall

Retrieval practice improves memory compared to strategies such as restudying and is thought to do so in at least two ways — the direct testing effect (improved memory for tested material) and the forward testing effect (improved memory for material learned after a practice test). Recent research implies that many studies of “the testing effect”, though usually defined in terms of the direct testing effect, conflate direct and forward testing effects (Mulligan et al., 2022). This motivates a new meta-analysis on testing effects with free recall that distinguishes between studies capable of isolating the direct effect of testing (direct studies) from those susceptible to both direct and forward testing effects (mixed studies). The results show that the large majority (67%) of the “testing effects” are from mixed studies, confounding direct and forward testing effects, and only 33% are from direct studies, with mixed studies producing larger testing effects than direct studies. Furthermore, the mixed studies differed in terms of whether the purported forward testing contribution should act to increase, decrease, or have an intermediate effect on the observed testing effect. The effect sizes differed across the mixed subtypes confirming this characterization of the mixed effects as a confounded mixture of forward and direct testing effects. Separate moderation analyses were conducted for the direct and mixed effects further indicating differences between the two population of effects. The literature on the testing effect is not a coherent body and does not clearly tell us about a specific memory phenomenon. Rather, it is a mixture of direct and forward testing effects, in which the forward component sometimes increases, sometimes decreases, and sometimes has an intermediate influence on the observed testing effect.