Leveraging Multi-Task Transfer Learning for Enhanced Breast Mass Classification in Digital Mammography

Purpose: Accurate breast mass classification is crucial for early breast cancer diagnosis. Deep learning shows promise in computer-aided diagnosis but faces challenges due to limited annotated data and lesion complexity. We propose a novel multi-task transfer learning framework to improve mass classification performance and provide a well-performed framework for medical image analysis. Methods The proposed framework comprises a transfer learning backbone and multi-task-specific branches. Pretraining weights from natural image datasets are leveraged to finetune the backbone network, enhancing the ability to extract breast mass characteristics. The classification branches include the primary task for breast mass classification and auxiliary tasks for BI-RADS evaluation, guiding the model to focus on relevant diagnostic features. Result Three models were compared using ResNet50/InceptionV3 as backbones. The Multi-Task Transfer Learning Framework (MTL + TL) achieved the highest AUC values of 0.852 ± 0.019 / 0.824 ± 0.021, outperforming other models in accuracy (0.7654 ± 0.0218 / 0.7667 ± 0.0224), precision (0.6842 ± 0.0379 / 0.7179 ± 0.0379), and F1-score (0.6842 ± 0.0379 / 0.7179 ± 0.0379). Grad-CAM heatmaps confirmed its effectiveness and ability to capture breast mass characteristics. Conclusion Our proposed framework significantly improved breast mass classification, addressing limited annotated data and providing an effective solution to address the limited availability of annotated data. The proposed framework enhanced feature recognition and overall performance by simulating a clinician's observation and decision-making of imaging features through the auxiliary task. This approach can be extended to other breast lesion classifications and provide valuable guidance for medical imaging analysis.