mnDINO: Accurate and robust segmentation of micronuclei with vision transformer networks

Abstract Recent advances in cell segmentation successfully produce models that generalize across various cell-lines and imaging types. However, these methods still fail to recognize subcellular structures such as micronuclei (MN), which are rare and tiny DNA-containing structures found outside of the main nucleus and observable under the microscope. While they can be hard to recognize in images, studying MN formation is of great interest because of their relationship to chromosome instability, genotoxicity, and cancer progression. Here we present a segmentation model, mnDINO, to segment micronuclei in DNA stained images under diverse experimental conditions with very high efficiency and accuracy. To train this model, we collected a heterogeneous set of images with more than five thousand annotated micronuclei. Trained with this diverse resource, the mnDINO model improves the accuracy of MN segmentation, and exhibits strong generalization across microscopes and cell lines. The dataset, code, and pre-trained model are made publicly available to facilitate future research in MN biology. Competing Interest Statement The authors have declared no competing interest. Footnotes Added funding sources, author contributions, competing interests statement at the end of the manuscript. Also, added additional details in the supplementary material and updated the supplementary figures. Finally, applied the following minor edits: citation style changed, clarifications in the caption of figure 5. https://www.ebi.ac.uk/biostudies/bioimages/studies/S-BIAD2809