Inconsistent Application of the Reliable Change Index in Clinical Psychology Trials Leads to Noisy Estimates of Individual Change

Objective: Clinical trials play an essential role in evaluating psychological interventions. Owing to limitations of standard analysis of average group-level change when conducted in isolation, it is common for clinical trials to also assess treatments regarding their ability to produce meaningful change for individual patients, known as clinical significance. One feature of clinical significance is the assessment of reliable change, which determines, for each individual, whether they have shown sufficient change to rule out measurement error. However, numerous approaches to assess reliable change have been proposed, and popular methods are underspecified, leading to the potential for inconsistent evaluation of reliable change across studies. Such inconsistencies may render reliable change results incomparable across studies. Methods: Using a methodological scoping review of four clinical psychology journals published between 2020 and 2023, this study evaluated how reliable change is typically operationalized in psychology clinical trials and assessed the impact of different operationalizations on reliable change estimates. Results: Among 226 clinical trials examined, 29 (12.8%) formally assessed reliable change. We identified at least seven distinct operationalizations of this construct, including five distinct operationalizations of the same formula from Jacobson & Truax (1991). Using example data to compute reliable change using each of these operationalizations, the reliable change threshold differed from between 7% and 79%, depending on the specific operationalization. Conclusion: Results show that implementation of reliable change is highly inconsistent in clinical trials, including when ostensibly using the same formula, harming comparability of results across studies. Recommendations are provided.