Label2label: Training a neural network to selectively restore cellular structures in fluorescence microscopy

Fluorescence microscopy is an essential tool in cell biology to visualise the spatial distribution of proteins that dictates their role in cellular homeostasis, dynamic cellular processes, and dysfunction during disease. However, unspecific binding of the antibodies that are used to label a cellular target often leads to high background signals in the images, decreasing the contrast of a cellular structure of interest. Recently, convolutional neural networks (CNNs) have been successfully employed for denoising and upsampling in fluorescence microscopy, but current image restoration methods cannot correct for background signals originating from the label. Here, we report a new method to train a CNN as content filter for non-specific signals in fluorescence images that does not require a clean benchmark, using dual-labelling to generate the training data. We name this method label2label (L2L). In L2L, a CNN is trained with image pairs of two non-identical labels that target the same cellular structure of interest. We show that after L2L training a network restores images not only with reduced image noise but also label-induced unspecific fluorescence signal in images of a variety of cellular structures, resulting in images with enhanced structural contrast. By implementing a multi-scale structural similarity loss function, the performance of the CNN as a content filter is further enhanced, for example, in STED images of caveolae. We show evidence that, for this loss function, sample differences in the training data significantly decrease so-called hallucination effects in the restorations that we otherwise observe when training the CNN with images of the same label. We also assess the performance of a cycle generative adversarial network as a content filter after L2L training with unpaired image data. Lastly, we show that a CNN can be trained to separate structures in superposed fluorescence images of two different cellular targets, allowing multiplex imaging with microscopy setups where the number of excitation sources or detectors is limited.