Hanging the Anchor Off a New Ship: A Meta-Analysis of Anchoring Effects

Anchoring effects are among the largest and easiest to replicate in social psychology. However, the ship on which anchoring research is floating has become brittle with respect to three domains: (a) the relationship between paradigm features (e.g., choice of anchor or type of scale) and effect sizes is mostly unknown, (b) there are numerous contradictory findings on moderators (e.g., role of incentives for accurate estimates), and (c) it is unclear which of the many theories on anchoring effects is best supported by the evidence overall. Method: To increase clarity and transparency in the field of anchoring effects research, we meta-analyzed the most comprehensive collection of openly available anchoring effects data in existence today. Results: We found large but also heterogeneous anchoring effects, g = 0.689, 95% CI [0.589, 0.789], σ² = 0.21, Ntotal = 17,708, k = 393. (a) Anchors communicated as randomly determined did not have effects, and some effects reversed when visual response scales were used. (b) For many previously contradictory results, we found null effects (e.g., incentives did not affect the magnitude of anchoring effects). (c) Overall, theories of anchoring that stress rationality and nonlinearity were best supported by the results. Discussion: By applying an open and robust empirical procedure, we provide anchoring researchers with a new ship. We recommend that future research increase trust in findings through preregistration, reduce heterogeneity between anchoring effects through standardization, and better explain the impact of moderators. Data and analyses are available online (https://osf.io/4t5mu/).