Using GAMMs to model trial-by-trial fluctuations in experimental data: More risks but hardly any benefit

Data from each subject in a repeated-measures experiment form a time series, which may include trial-by-trial fluctuations arising from human factors such as practice or fatigue. Concerns about the statistical implications of such effects have increased the popularity of Generalized Additive Mixed Models (GAMMs), a powerful technique for modeling wiggly patterns. We question these statistical concerns and investigate the costs and benefits of using GAMMs relative to linear mixed-effects models (LMEMs). In two sets of Monte Carlo simulations, LMEMs that ignored time-varying effects were no more prone to false positives than GAMMs. Although GAMMs generally boosted power for within-subject effects, they reduced power for between-subject effects, sometimes to a severe degree. Our results signal the importance of proper subject-level randomization as the main defense against statistical artifacts due to by-trial fluctuations.