Characterizing the nonlinear structure of shared variability in cortical neuron populations using latent variable models

Sensory neurons often have variable responses to repeated presentations of the same stimulus, which can significantly degrade the information contained in those responses. Such variability is often shared across many neurons, which in principle can allow a decoder to mitigate the effects of such noise, depending on the structure of the shared variability and its relationship to sensory encoding at the population level. Latent variable models offer an approach for characterizing the structure of this shared variability in neural population recordings, although they have thus far typically been used under restrictive mathematical assumptions, such as assuming linear transformations between the latent variables and neural activity. Here we leverage recent advances in machine learning to introduce two nonlinear latent variable models for analyzing large-scale neural recordings. We first present a general nonlinear latent variable model that is agnostic to the stimulus tuning properties of the individual neurons, and is hence well suited for exploring neural populations whose tuning properties are not well characterized. This motivates a second class of model, the Generalized Affine Model, which simultaneously determines each neuron’s stimulus selectivity and a set of latent variables that modulate these stimulus responses both additively and multiplicatively. While these approaches can detect general nonlinear relationships in shared neural variability, we find that neural activity recorded in anesthetized primary visual cortex (V1) is best described by a single additive and single multiplicative latent variable, i.e., an “affine model”. In contrast, application of the same models to recordings in awake macaque prefrontal cortex discover more general nonlinearities to compactly describe the population response variability. These results thus demonstrate how nonlinear latent variable models can be used to describe population variability, and suggest that a range of methods is necessary to study different brain regions under different experimental conditions.