Physics-Informed Generative Model for 3D Localization Microscopy

Abstract Localization microscopy has overcome the diffraction limit, i.e. the conventional resolution limit of a microscope, enabling nanoscale biological imaging by precisely determining the positions of individual emitters such as single fluorescent molecules. However, the performance of deep learning methods, commonly applied to these tasks, depends significantly on the quality of training data, typically generated through simulation. Creating simulations that perfectly replicate experimental conditions remains challenging, resulting in a persistent simulation-to-experiment gap. To bridge this gap, we propose a physics-informed generative model leveraging self-supervised learning directly on experimental data. Our model extends the Deep Latent Particles (DLP) framework by incorporating a physical model of the Point Spread Function (PSF; the image of a single point source in the microscope) into the decoder, enabling it to disentangle learned realistic environments from emitters. Trained directly on unlabeled experimental images, our model intrinsically captures realistic background, noise patterns, and emitter characteristics. The decoder thus acts as a high-fidelity generator, producing fully labeled, realistic training images with known emitter locations. Using these generated datasets significantly improves the performance of supervised localization networks, particularly in challenging scenarios such as complex backgrounds and low signal-to-noise ratios. We demonstrate our approach on a variety of experimentally measured microscopy data, including super-resolution imaging in 2D and 3D and particle tracking in live cells, showing substantial improvements in localization precision and emitter detection. The code will be made publicly available. Competing Interest Statement The authors have declared no competing interest. Footnotes ofrigoldenberg{at}campus.technion.ac.il Title updated to include "Self-Supervised." Abstract revised and keywords added. Related work expanded with detailed analysis of why other generative approaches are fundamentally incompatible with sub-pixel SMLM precision. Figure 1 redesigned to better illustrate the PILPEL use case and generated data pipeline. Section 5.2 (super-resolution 3D imaging) revised: baseline comparison updated to a new experimental dataset trained on standard DS3D, with detections increasing from 119K to 581K (∼4.9x); discussion of density filtering and false positives added. Two new experiments added (Section 5.4): 2D super-resolution reconstruction of microtubules using DeepSTORM, and 3D localization in bacteria using astigmatism PSF, demonstrating generalizability across PSF types and biological systems. Precision-Recall analysis added to main text (Section 5.1, Fig. 3c). Ablation study added (Supplementary Sec. D). Noise latent variable explicitly described in model architecture. Limitations and conclusions sections revised. Comprehensive supplementary material added, including experimental protocols, computational details, hyperparameter tables, and additional reconstructions. References expanded.