Nuclei Segmentation in Hematoxylin & Eosin- stained Tissue Images Using U-Net-based Models

Digital pathology is a rapidly evolving field that seeks to automate workflows to provide clinically relevant information. Automated segmentation algorithms aid in providing quantitative features, such as size and shape of the nuclei, essential in diagnosing diseases. These algorithms’ accuracy is critical for generating reliable results with minimal user intervention while significantly reducing the time required to analyze tissue samples. Deep learning-based methods have become popular for their superior performance in segmentation tasks. We propose a nuclei segmentation technique using a U-Net-based model in this work. A U-Net-based model was given two decoder units, one for binary mask prediction and another for ternary mask prediction, and a pre-trained VGG-19 to be used as the encoder. The final instance segmentation masks are then created using a watershed algorithm based on the nuclear markers extracted from ternary masks. We perform this technique using U-Net and its more recent variants, such as U-Net ++, U-Net 3+, and Double U-Net, to compare their performances. Experimental results on the publicly available MoNuSeg dataset show that the proposed method using the Double U-Net model achieves an average aggregate Jaccard index of 67.61%, outperforming several state-of-the-art algorithms applied on the same dataset.