A Note on the Structural Change Test in Highly Parameterized Psychometric Models

Equal parameter estimates across subgroups is a substantial requirement of statistical tests. Ignoring subgroup differences poses a threat to study replicability, model specification, and theory development. One powerful statistical method that allows testing for parameter invariance is structural change tests. A core element of those tests is the empirical fluctuation process. In the case of parameter invariance, the fluctuation process asymptotically follows a Brownian bridge. This asymptotic assumption further provides the basis for inference. However, in this paper, we show that the empirical fluctuation process does not follow a Brownian bridge in small samples. Thus, methods of obtaining the sampling distribution are incorrect, and the p-value misspecified. Therefore, we implement an alternative solution to obtaining the sampling distribution - permutation approaches. Permutation approaches obtain the sampling distribution through resampling of the dataset, avoiding unmet distributional assumptions. We show that the permutation approach solves the issue of the misspecified sampling distribution and increases power, therefore serves as a superior method to standard asymptotic approximations of the sampling distribution.