Brain2Pix: Fully convolutional naturalistic video reconstruction from brain activity

Lynn Le, Luca Ambrogioni, Katja Seeliger, Yağmur Güçlütürk, Marcel van Gerven, Umut Güçlü
2021 · doi:10.1101/2021.02.02.429430
preprint OA: closed
📄 Open PDF View at publisher

Abstract

Reconstructing complex and dynamic visual perception from brain activity remains a major challenge in machine learning applications to neuroscience. Here we present a new method for reconstructing naturalistic images and videos from very large single-participant functional magnetic resonance imaging data that leverages the recent success of image-to-image transformation networks. This is achieved by exploiting spatial information obtained from retinotopic mappings across the visual system. More specifically, we first determine what position each voxel in a particular region of interest would represent in the visual field based on its corresponding receptive field location. Then, the 2D image representation of the brain activity on the visual field is passed to a fully convolutional image-to-image network trained to recover the original stimuli using VGG feature loss with an adversarial regularizer. In our experiments, we show that our method offers a significant improvement over existing video reconstruction techniques.

My notes (saved in your browser only)

Citation neighborhood (sparse)

Too few in-corpus citations on either side for a chart; here are the lists.

Cites (2)

References (40)

Source provenance

crossref
last seen: 2026-07-03T06:29:24.753609+00:00
europepmc
last seen: 2026-05-19T01:45:01.086888+00:00
unpaywall
last seen: 2026-07-09T06:39:34.564547+00:00