An Anthropomorphic Diagnosis System of Pulmonary Nodules using Weak Annotation-Based Deep Learning

Purpose To develop an anthropomorphic diagnosis system of pulmonary nodules (PN) based on Deep learning (DL) that is trained by weak annotation data and has comparable performance to full-annotation based diagnosis systems. Methods The proposed system uses deep learning (DL) models to classify PNs (benign vs. malignant) with weak annotations, which eliminates the need for time-consuming and labor-intensive manual annotations of PNs. Moreover, the PN classification networks, augmented with handcrafted shape features acquired through the ball-scale transform technique, demonstrate capability to differentiate PNs with diverse labels, including pure ground-glass opacities, part-solid nodules, and solid nodules. Results The experiments were conducted on two lung CT datasets: (1) public LIDC-IDRI dataset with 1,018 subjects, (2) In-house dataset with 2740 subjects. Through 5-fold cross-validation on two datasets, the system achieved the following results: (1) an Area Under Curve (AUC) of 0.938 for PN localization and an AUC of 0.912 for PN differential diagnosis on the LIDC-IDRI dataset of 814 testing cases, (2) an AUC of 0.943 for PN localization and an AUC of 0.815 for PN differential diagnosis on the in-house dataset of 822 testing cases. These results demonstrate comparable performance to full annotation-based diagnosis systems. Conclusions Our system can efficiently localize and differentially diagnose PNs even in resource-limited environments with good robustness across different grade and morphology sub-groups in the presence of deviations due to the size, shape, and texture of the nodule, indicating its potential for future clinical translation. Summary An anthropomorphic diagnosis system of pulmonary nodules (PN) based on deep learning and weak annotation was found to achieve comparable performance to full-annotation dataset-based diagnosis systems, significantly reducing the time and the cost associated with the annotation. Key Points A fully automatic system for the diagnosis of PN in CT scans using a suitable deep learning model and weak annotations was developed to achieve comparable performance (AUC = 0.938 for PN localization, AUC = 0.912 for PN differential diagnosis) with the full-annotation based deep learning models, reducing around 30%∼80% of annotation time for the experts. The integration of the hand-crafted feature acquired from human experts (natural intelligence) into the deep learning networks and the fusion of the classification results of multi-scale networks can efficiently improve the PN classification performance across different diameters and sub-groups of the nodule.