A call for greater transparency in piloting

A pilot study is a preliminary investigation conducted before a full-scale research study. Although piloting is a common and valuable part of the research process, pilot studies are not systematically reported (or even acknowledged) in the psychology literature. This paper argues that establishing norms and guidelines for regularly reporting pilot studies can improve many aspects of psychological research. Specifically, reporting pilot studies can help researchers avoid systematically excluding part of the research process from the scientific record, reveal selection biases in research designs that may affect the generalizability of findings, evaluate others’ research more accurately, learn from other researchers’ successes and failures, uphold ethical responsibilities, and improve the methodological practice of piloting. We conclude with three concrete suggestions for near-term changes to help pilot reporting become standard practice: scientific communities and institutions should develop simple pilot reporting templates; journals should establish clear pilot reporting guidelines for authors; and authors should acknowledge when pilot studies were run and share relevant materials and data. Current reporting standards favor conclusive and polished research narratives, but such narratives are the end product of a complex scientific process that should also be shared. A truly transparent science should spotlight processes, like piloting, that underpin the conclusions scientific papers draw.