A Practical Guide to Estimating Reliability of Intensive Longitudinal Data

This practical guide provides an overview of different methods that can be used to assess thereliability of intensive longitudinal data. Intensive longitudinal data, which involve manyconsecutive measurements of individual experiences, offer new perspectives into the within-person variability of psychological constructs. Unfortunately, the reliability of these data isoften neglected. Yet, assessing the reliability of intensive longitudinal data measurements iscrucial, as unreliable measurements can lead to inaccurate and imprecise results. Variousmethods for assessing reliability of intensive longitudinal data have been recently developed.These include methods based on well-known frameworks of test-retest reliability – suitable forsingle-item constructs – internal consistency methods, and parallel-test methods. However,many of these may be unknown to empirical researchers, and each method for evaluatingreliability has its own assumptions, interpretations, and practical considerations. In thisoverview we will discuss the available methods and these considerations. Moreover, we applythe methods to an empirical dataset to highlight their similarities, differences, and context-dependent utility. We have implemented the available methods in R code that researchers canuse for their own data. Reliability assessment in ambulatory studies requires carefulconsideration of both methodological assumptions and practical constraints. With this guidewe hope to aid researchers in this process.