Many analysts and few incentives

The Many Analysts Religion Project (MARP) follows in a tradition of many analysts projects that have examined how variations in analytic choices can affect results (Silberzahn & Uhlmann, 2015). One of the core strengths of the many analysts approach is that there are no direct incentives for analysis teams to attempt to achieve statistical significance (Silberzahn et al., 2018). However, in this commentary we suggest that a lack of direct incentives to achieve particular outcomes can be a double-edged sword. We use our own submissions to the MARP project to illustrate our point.