Sensory Modality- and Load-Dependent Changes across Cortical Representations for Working Memory Storage

While distributed cortical areas represent working memory contents, their necessity for memory maintenance has been questioned. We examined the differential effects of maintaining multiple items on the neural information across cortical regions. In each trial of the fMRI experiment, participants (n=81) had to memorize two items, each either an orientation or a pure pitch, for 13.8s and continuously recalled the target after the delay. We kept the overall working memory load constant, but varied the sensory modality of each item to vary the effective visual load. We show that increasing visual load decreased behavioural recall performance for orientations and continuous orientation-specific decodable information in visual cortex but less so infrontoparietal areas. Simulations show that this drop in decodable information is best interpreted as a drop in mnemonic information represented by patterns of visual cortex activity. Our results provide evidence for shared labour of visual cortices, where maintaining two versus one orientation leads to a loss in representational fidelity, and anterior cortices, where multiple items could be represented in a more robust but less precise format.