Interdependence of ‘what’ and ‘when’ in the brain

From a brain’s-eye-view, when a stimulus occurs, and what it is, are interrelated aspects of interpreting the perceptual world. Yet in practice the putative perceptual inferences about sensory content and timing are often dichotomized and not investigated as an integrated process. We here argue that neural temporal dynamics can influence what is perceived, and in turn, stimulus content can influence the time at which perception is achieved. This computational principle results from the highly interdependent relationship of ‘what’ and ‘when’ in the environment. Both brain processes and perceptual events display strong temporal variability which is not always modeled; understanding - and minimally, modeling - this temporal variability is key for theories of how the brain generates unified neural representations. Ignoring the presence of temporal variability can lead to ill-formed data interpretations. Here, we review ‘what’ and ‘when’ interactions in the brain, demonstrate via simulation how temporal variability can result in misguided conclusions, and outline how to integrate and synthesize ‘what’ and ‘when’ in theories and models of brain computation.