Ubiquitous bias & false discovery due to model misspecification in analysis of statistical interactions: The role of the outcome's distribution and metric properties

Lived experience suggests substantial heterogeneity in how people react to a common stimuli. Previous work has considered several potential sources of bias and confusion in studying interactions but less attention has been devoted to the nature of the outcome variable in such studies. Here, we consider bias and false discovery associated with estimates of interaction parameters as a function of the distributional and metric properties of the outcome. Focusing on a variety of models for non-continuously distributed outcomes (binary and count outcomes), we show that attempts to use the linear model for recovery lead to catastrophic levels of bias and false discovery. Focusing on transformations of normally distributed variables (i.e., censoring and departures from interval scaling), we show that linear models again produce spurious interaction effects when such interactions are absent from the generating model. We provide explanations offering geometric and algebraic intuition as to why interactions can be a challenge for these incorrectly specified models. In light of these findings, we make two specific suggestions. First, a careful consideration of the outcome's distributional properties should be a standard component of interaction studies. Second, researchers should approach research focusing on interactions with heightened levels of scrutiny.