Ten Strategies to Improve Many-Analysts Studies

The many-analysts approach has demonstrated a surprising diversity in analytic choices and outcomes given a single research question and dataset. As a result, the approach has gained traction as a tool for revealing and understanding epistemic uncertainty. Despite its growing popularity, there is still limited practical guidance for team leaders on how to execute such studies. In this perspective, we propose ten strategies to improve the many-analysts methodology. The proposed strategies were build based on insights from past projects and our own experience as team leaders and are organized around six themes: (1) moving many-analysts projects beyond merely exposing analytic variability; (2) creating structured opportunities for participating teams to share their insights; (3) recognizing analytical quality and expertise; (4) uncovering sources of variability through meta-regression and complementary methods; (5) addressing epistemic uncertainty on a smaller scale; and (6) developing comprehensive models of uncertainty. To showcase their practical application, we illustrate these strategies using data from the Many-Analysts Religion Project.