Physics-based Deep Learning for Imaging Neuronal Activity via Two-photon and Light Field Microscopy

Light Field Microscopy (LFM) is an imaging technique that offers the opportunity to study fast dynamics in biological systems due to its rapid 3D imaging rate. In particular, it is attractive to analyze neuronal activity in the brain. Unlike scanning-based imaging methods, LFM simultaneously encodes the spatial and angular information of light in a single snapshot. However, LFM is limited by a trade-off between spatial and angular resolution and is affected by scattering at deep layers in the brain tissue. In contrast, two-photon (2P) microscopy is a point-scanning 3D imaging technique that achieves higher spatial resolution, deeper tissue penetration, and reduced scattering effects. However, point-scanning acquisition limits the imaging speed in 2P microscopy and cannot be used to simultaneously monitor the activity of a large population of neurons. This work introduces a physics-driven deep neural network to image neuronal activity in scattering volume tissues using LFM. The architecture of the network is obtained by unfolding the ISTA algorithm and is based on the observation that the neurons in the tissue are sparse. The deep-network architecture is also based on a novel imaging system modeling that uses a linear convolutional neural network and fits the physics of the acquisition process. To achieve the high-quality reconstruction of neuronal activity in 3D brain tissues from temporal sequences of light field (LF) images, we train the network in a semi-supervised manner using generative adversarial networks (GANs). We use the TdTomato indicator to obtain static structural information of the tissue with the microscope operating in 2P scanning modality, representing the target reconstruction quality. We also use additional functional data in LF modality with GCaMP indicators to train the network. Our approach is tested under adverse conditions: limited training data, background noise, and scattering samples. We experimentally show that our method performs better than model-based reconstruction strategies and typical artificial neural networks for imaging neuronal activity in mammalian brain tissue, considering reconstruction quality, generalization to functional imaging, and reconstruction speed.