Learners do not readily form abstract category representations across sensory modalities

A longstanding debate in cognitive science centers around whether our internal representations of the external world are abstract and amodal or whether they are grounded by sensory modality. Perceptual category learning presents a unique testbed of this debate as categories exist within all sensory modalities, but there is mixed evidence about whether people form abstract representations that can be analogized across modalities. In the current study, we examine whether learners form such abstract representations of auditory and visual categories that share the same rule-based structure. Across three experiments, we examine whether learners spontaneously or explicitly form abstract representations across sensory modalities. Overall, we found no evidence of shared abstract representations even in the conditions that would theoretically most strongly encourage it – when information across modalities was conceptually aligned (rather than misaligned), when information across modalities was encountered within the same task (rather than separate tasks), and when participants were explicitly instructed to analogize across modalities (rather than doing so spontaneously). These results suggest that even when abstract representation across auditory and visual modalities is encouraged, the knowledge formed during perceptual category learning is modality specific. These results highlight the importance of considering the role of modality in category learning and cognition more broadly.