Seeing sounds: Neural mechanisms underlying auditory contributions to visual detection

Sounds enhance the detection of visual stimuli while concurrently biasing an observer’s decisions. To investigate the neural mechanisms that underlie such multisensory interactions, we decoded time-resolved signal detection theory (SDT) sensitivity and criterion parameters from neural activity using magnetoencalography, while participants performed a visual detection task. Using temporal generalization analysis, we found that sounds improve visual detection by enhancing the maintenance of the most informative perceptual samples over time. In parallel, criterion decoding analyses revealed that sounds evoke patterns of activity that resembled the patterns evoked by an actual visual stimulus. These two complementary mechanisms of audiovisual interaction differed in terms of their automaticity: Whereas the sound-induced enhancement in visual information maintenance depended on participants being actively engaged in a detection task, sounds evoked visual activity patterns in the visual cortex in a bottom-up fashion, challenging the classical assumption that sound- induced reductions in criterion correspond to decision-level biases.