Measurement Invariance Violation Indices (MIVIs): Effect sizes for (partial) non-invariance of items and item sets

In many applications, measurement invariance does not hold. When a model with a certain level of invariance is rejected, the amount of non-invariance bias may either be consequential or practically irrelevant. So far, few attempts have been made to quantify the extent of bias due to the lack of measurement invariance. We derived new effect sizes from first principles called Measurement Invariance Violation Indices (MIVIs) for items and item sets. MIVIs assume that one can compare the basic measurement model across groups (i.e., configural invariance holds) but cannot compare some factor loadings, intercepts, and/or unique variances. Assuming partial invariance for a set of items, MIVIs quantify non-invariant factor loading, intercept, or uniqueness differences in relation to the pooled latent standard deviation (either per item or as an average for item sets). Thus, parameter differences can be interpreted in standard deviation units (of the pooled latent variable). One can further inspect the compensatory cancelation and non-compensatory aggregation effects of non-invariance bias when maintaining the directional information (signed MIVIs). MIVIs support the group-fair item selection, help to evaluate the questionnaire quality, and allow for assessing the amount of non-invariance bias in index scores (e.g., mean scores).