Comparing researchers’ degree of dichotomous thinking using frequentist versus Bayesian null hypothesis testing

A large amount of scientific literature in social and behavioural sciences bases their conclusions on one or more hypothesis tests. As such, it is important to obtain more knowledge about how researchers in social and behavioural sciences interpret quantities that result from such hypothesis tests, such as p-values and Bayes factors. In the present study, we will explore the relationship between obtained statistical evidence and the degree of belief or confidence that there is a positive effect in the population of interest. In particular, we are interested in the existence of a so-called cliff effect: A qualitative drop in the degree of belief that there is a positive effect around certain threshold values of statistical evidence. We will compare this relationship for p-values to the relationship for corresponding degrees of evidence quantified through Bayes factors, and we will examine whether this relationship is affected by two different modes of presentation.