Sensorimotor Account of Multimodal Prosody

The temporal signatures that characterize speech—especially its prosodic qualities—are observable in the movements of the hands and bodies of its speakers. A neurobiological account of these prosodic rhythms is thus likely to benefit from insights on the neural coding principles underlying co-speech gestures. Here we consider a role for the information processing capacity of the vestibular system, a sensory system that encodes movements of the body, in the neural representation of prosody. Careful review of the vestibular system’s functional organization as a hierarchical and predictive system, its relevance for rhythmic sound sequences, dynamic attention, and its involvement in vocalization suggests vestibular codes may be important for the neural tracking of speech. As the kinematics and time course of co-speech movements mirror prosodic fluctuations in spoken prosody, we argue that the vestibular system helps encode and decode these prosodic features in multimodal discourse and possibly during auditory speech processing alone.