A CoGenT account of a Compositional and Generative Theory for the basis of working memory

The typical conception of working memory is a mechanism to temporarily hold multiple discrete objects in service of other cognitive tasks. In this paper, we elaborate on this perspective by proposing that generative and compositional processing form the functional basis of working memory, especially in the visual form. Compositionality allows complex scenes or objects to be mentally decomposed into constituents that can be individually manipulated or recombined to form new representations. Generative processing allows purely conceptual information to be reconstructed in a format akin to visual sensory representations that can be manipulated and re-processed by perceptual mechanisms. Together, compositional and generative mechanisms would enable a wide range of cognitive functions including the basis of visual imagery. We also describe a more flexible role for memory slots, which are often assumed as a core capacity of working memory. In this view, slots in working memory are typically used to support compositional processing of scenes or complex objects. We conclude with a conceptual account of such a memory system with an emphasis on composition by parts or features. This model is linked to a variety of experimental results from the memory and visual imagery literatures that illustrate the flexibility of such a system for performing cognitive tasks.