A note on type S/M errors in hypothesis testing

Motivated by the recent replication and reproducibility crisis, Gelman and Carlin (2014) advocated focusing on controlling for type S/M errors, instead of the classic type I/II errors, when conducting hypothesis testing. In this paper, we aim to fill several theoretical gaps in the methodology proposed by Gelman and Carlin (2014). In particular, we derive the closed-form expression for the expected type M error, and study the mathematical properties of the probability of type S error as well as the expected type M error, such as monotonicity. We demonstrate the advantages of our results through numerical and empirical examples.