Signposts on the Path from Nominal to Ordinal Scales

Polytomous data are often categorized as either ordinal or nominal. However, the nominal versus ordinal distinction need not be viewed as a dichotomy; here, we frame it as a continuum. This framing necessitates the use of indices that are informative about the location of data in that continuum. In this study, we consider six such indices and apply them to simulated and empirical item response data. We evaluate the degree to which these indices quantify category order using item response models to generate data with qualitatively distinctive properties and compare them with respect to their behavior and computational demand. We find that two of the parametric indices (Index 5: Mean Difference between Slope Parameters and Index 6: Arctangent of Paired Category Ratios) show the most potential in theory—they are robust to low-frequency categories—and interpretation—positive and negative values indicate order and disorder, respectively. The rest of the indices are more sensitive to response frequencies and provide relative measures of order within a test. Nonparametric indices have the advantage of lower computational time. These indices could be used to help find and apply more suitable IRT models, resulting in more accurate latent variable and item parameter estimates. They can also inform us of how changing model parameters translates to the ordinal properties of generated item response data.