Noise in the process: Assessing the evidential value of mediation analyses in marketing and consumer psychology journals

Mediation analysis plays a central role in marketing research due to its usefulness in helping to explain complex processes. Like other forms of inference, mediation analyses are susceptible to false positive results. This is particularly true when analytic decisions are based on the data, rather than a priori hypotheses. To assess the collective evidential value of mediation analyses in marketing, we used an approach first implemented by Götz and colleagues (2021) that (1) measures the relative proximity of confidence intervals to zero (RP) and (2) aggregates a related set of RP scores into a single distribution. For our analysis, we compared the RP distribution of top marketing journals (2018-20) to simulations of low power, adequate power, and null effects. We also compared the marketing journals to real-world data from Journal of Personality and Social Psychology (JPSP) (2018-20). We found that, in terms of evidential value, mediation analyses in marketing substantially deviated from simulations of adequate power and JPSP but were similar to simulations of low power and null effects. We propose study preregistration, corrections for multiple testing, and increased statistical power as solutions to increase evidence quality going forward.