Meta-regression sensitivity to study miscategorisation: Implications and recommendations for double-coding in meta-analysis

Meta-analysis enables researchers to summarise the empirical literature on aphenomenon. In some cases, meta-analysts also conduct theoretically-motivated moderatoranalyses, or meta-regressions, to determine whether the magnitude of an effect depends onrelevant study characteristics (e.g., the type of stimuli or measure used). To do so requires firstmanually extracting information from, or coding, the meta-analysed studies. Although theveracity of moderator analyses depends on the accuracy of the coding process, many researchersopt to double-code only a fraction of their studies, leaving open the possibility of coding errorsin the remaining data. Here, we simulated the impact of seemingly low rates (.02, .04, and .06) ofrandom (Study 1) and biased (Study 2) study miscategorisation on meta-regression results.Results indicated that miscategorisation had a larger effect in smaller meta-analyses and whenmoderator categories were imbalanced, even when the miscategorisation rate was as low as .02.We propose a set of guidelines to enhance the efficacy of partial double-coding and mitigatesuch effects when complete double-coding is infeasible.