Whether the Pairwise Rating Method and the Spatial Arrangement Method Yield Comparable Dimensionalities Depends on the Dimensionality Choice Procedure

We investigate whether two methods for obtaining similarity data yield multidimensional scaling (MDS) solutions of comparable dimensionality. In the Pairwise Rating Method (PRaM), participants rate the (dis)similarity of all pairs of stimuli on a Likert scale. In the Spatial Arrangement Method (SpAM), participants organize stimuli on a computer screen so that the distance between stimuli represents their perceived dissimilarity. Across two studies that included eight semantic categories with varying numbers of both pictorial and verbal exemplars, we did not find consistent dimensionality differences between the two similarity measurement methods. The results alleviate the concern that because of its two-dimensional nature, SpAM might underestimate the dimensionality of high-dimensional stimuli compared to PRaM. Aggregating the SpAM similarity data from a sufficient number of participants can yield spatial representations with more than two dimensions. However, the resulting number of dimensions was found to be highly dependent on the dimensionality choice procedure. Even for specific combinations of a single category and similarity measurement method, different dimensionalities were obtained depending on whether the reliability of the similarity data, Monte Carlo simulations, or predictive correlations were used to establish the number of dimensions, indicating the need for a more systematic investigation into dimensionality selection for MDS.