How do scientists update their beliefs? An investigation of N scientists engaged in replication research

How, if at all, do scientists adjust their beliefs based on observed evidence? Few studies have examined how beliefs change over the course of a study, from developing the protocol to collecting and analyzing original data. N scientists who were contributing to one or more of six multi-lab replication projects were asked to estimate their degree of belief in the original effect and to estimate the true effect size at three timepoints: before data collection, after learning their own results, and after learning the results of all of the other laboratories contributing replication studies. We examine how beliefs changed in response to data, whether scientists are disproportionately influenced by their own study results relative to those of other labs, and whether people optimally update their prior beliefs according to the observed evidence.