Approaches for Quantifying the ICC in Multilevel Logistic Models: A Didactic Demonstration

Multilevel modeling techniques have gained traction among experimental psychologists for their ability to account for dependencies in nested data structures. Increasingly, these techniques are extended to the analysis of binary data (e.g., choices, accuracy). Despite their popularity, these logistic multilevel models are often underutilized when researchers focus solely on fixed effects and ignore important heterogeneity that exists within and between participants, the random effects. In this tutorial, we review four techniques for estimating and quantifying residual- and cluster-level variance in logistic multilevel models in an accessible manner using real data. First, we introduce logistic multilevel modeling, including the interpretation of fixed and random effects. Second, we review the challenges associated with the estimation and interpretation of within- and between-participant variation in logistic multilevel models, particularly computing the intraclass correlation coefficient (ICC), which is usually a first, simple step in a linear MLM. Third, we demonstrate four existing methods of quantifying within- and between-participant variation in logistic multilevel models and discuss their relative advantages and disadvantages. Fourth, we present bootstrapping methods to make statistical inference about these variance estimates. To facilitate reuse, we developed R code to implement the discussed techniques, which is provided throughout the text and as supplemental materials.