General claims require generalized effects: A reply to Ruiz et al.’s (2020) ‘A systematic and critical response to Pendrous et al. (2020) replication study’

Recent evidence suggests replications of psychology studies produce discrepant findings, and that original studies frequently report inflated estimates of effect size. Colloquially speaking, studies ‘fail to replicate’ at an alarming rate, at least in terms of congruence of finding significant p values. Replies are often then written by the authors of the original studies. This discourse often follows a predictable pattern: highly general claims are made in an original study, the replication produces null results, and the response by the original authors primarily focuses on reasons to disqualify the replication’s results from requiring consideration, rather than acknowledging that the original study’s finding may not, in fact, be replicable, or that the generality of the original claims may require revision or constraint. I illustrate these points using the example of a recently published trio of an original study, a failed replication, and a response by original authors. I argue that our scientific goals would be better served by efforts to avoid falling into these writing tropes and to instead move the discourse forward and reinforce the behaviours we want to see in our scientific community, such as the conduction of high-quality replication studies. All data and code at osf.io/du8xk.