Cutting through the noise: Auditory scenes and their effects on visual object processing

Despite the intuitive feeling that our visual experience is coherent and comprehensive, the world is full of ambiguous and indeterminate information. Here, we explore how the visual system might take advantage of ambient sounds to resolve this ambiguity. Participants were tasked with identifying an object slowly fading in through visual noise while a task-irrelevant sound played. Across five experiments, we found that participants demanded more visual information when the auditory object was incongruent with the visual object compared to when it was not. Auditory scenes, which are only probabilistically related to specific objects, produced similar facilitation even for unheard objects (eg, a bench). Notably, these effects traverse categorical and specific auditory and visual processing domains as participants performed across-category and within-category visual tasks; underscoring crossmodal integration across multiple levels of perceptual processing. Overall, our study reveals the importance of audio-visual interactions to support meaningful perceptual experiences in naturalistic settings.