A biologically-inspired hierarchical convolutional energy model predicts V4 responses to natural videos

SUMMARY V4 is a key area within the visual processing hierarchy, and it represents features of intermediate complexity. However, no current computational model explains V4 responses under natural conditions. To address this, we developed a new hierarchical convolutional energy (HCE) model reflecting computations thought to occur in areas V1, V2, and V4, but which consists entirely of simple- and complex-like units like those found in V1. In contrast to prior models, the HCE model is trained end-to-end on neurophysiology data, without relying on pre-trained network features. We recorded 313 V4 neurons during full-color nature video stimulation and fit the HCE model to each neuron. The model’s predicted optimal patterns (POPs) revealed complex spatiotemporal pattern selectivity in V4, supporting its role in representing space, time, and color. These findings indicate that area V4 is crucial for image segmentation and grouping operations that are essential for complex vision. Thus, responses of V4 neurons under naturalistic conditions can be explained by a hierarchical three-stage model where each stage consists entirely of units like those found in area V1.