Reliability and feasibility of Linear Mixed Models in fully crossed experimental designs

The use of Linear Mixed Models (LMMs) is increasing in Psychology and Neuroscience research. A key aspect of LMMs is choosing a random effects structure according to the experimental needs. To date, opposite suggestions are present in the literature, spanning from keeping all random effects, which produces several singularity and convergence issues and often requires high computational resources, to removing random effects until the best fit is found, with the risk of inflating type I error. However, defining the random structure to fit a non-singular and convergent model is not straightforward. Moreover, the lack of a standard approach may lead the researcher to make decisions that potentially inflate type I errors and generate distortions in the estimates. To date, how to deal with singular and non-converging models is an ongoing debate.We introduce a new way to control for type I error inflation during model reduction using complex random intercepts (CRIs). These are multiple random intercepts that represent the residual variance of categorical fixed effects within a given grouping factor. We validated CRIs and the proposed procedure by extensive simulations and a real-case application. We demonstrate that CRIs can produce reliable results and require less RAM memory and computational time. Moreover, we outline a few criteria and clear recommendations on how and when scholars should reduce singular and non-converging models. Overall, the proposed procedure provides clear solutions to avoid overinflated results using LMMs in Psychology and Neuroscience.