Simulation Studies as a Tool to Understand Bayes Factors

When social scientists wish to learn about an empirical phenomenon, they perform an experiment. When they wish to learn about a complex numerical phenomenon,they can perform a simulation study. The goal of this paper is twofold. Firstly, this paper introduces how to set up a simulation study using the relatively simple example of simulating from the prior. Secondly, this paper demonstrates how simulation can be used to learn about the Jeffreys-Zellner-Siow (JZS) Bayes factor: a currently popular implementation of the Bayes factor employed in the BayesFactor R-package and freeware program JASP. Many technical expositions exist on JZS Bayes factors, but these may be somewhat inaccessible to researchers that are not specialized in statistics. This paper aims to show in a step-by-step approach how a simple simulation script can be used to approximate the calculation of the JZS Bayes factor. We explain how a researcher can write such a sampler to approximate JZS Bayes factors in a few lines of code, what the logic is behind the Savage Dickey method used to visualize JZS Bayes factors, and what the practical differences are for different choices of the prior distribution for calculating Bayes factors.