DeepSpot: a deep neural network for RNA spot enhancement in smFISH microscopy images

Detection of RNA spots in single molecule FISH microscopy images remains a difficult task especially when applied to large volumes of data. The small size of RNA spots combined with high noise level of images often requires a manual adaptation of the spot detection thresholds for each image. In this work we introduce DeepSpot, a Deep Learning based tool specifically designed to enhance RNA spots which enables spot detection without need to resort to image per image parameter tuning. We show how our method can enable the downstream accurate detection of spots. The architecture of DeepSpot is inspired by small object detection approaches. It incorporates dilated convolutions into a module specifically designed for the Context Aggregation for Small Object (CASO) and uses Residual Convolutions to propagate this information along the network. This enables DeepSpot to enhance all RNA spots to the same intensity and thus circumvents the need for parameter tuning. We evaluated how easily spots can be detected in images enhanced by our method, by training DeepSpot on 20 simulated and 1 experimental datasets, and have shown that more than 97% accuracy is achieved. Moreover, comparison with alternative deep learning approaches for mRNA spot detection (deepBlink) indicated that DeepSpot allows more precise mRNA detection. In addition, we generated smFISH images from mouse fibroblasts in a wound healing assay to evaluate whether DeepSpot enhancement can enable seamless mRNA spot detection and thus streamline studies of localized mRNA expression in cells.