An integrated Framework for Breast Mass Classification and Diagnosis using Stacked Ensemble of Residual Neural Networks

An integrated Framework for Breast Mass Classification and Diagnosis using Stacked Ensemble of Residual Neural Networks | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article An integrated Framework for Breast Mass Classification and Diagnosis using Stacked Ensemble of Residual Neural Networks Asma Baccouche, Begonya Garcia-Zapirain, Adel Elmaghraby This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-1389924/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 8 You are reading this latest preprint version Abstract A computer-aided diagnosis (CAD) system requires automated stages of tumor detection, segmentation, and classification that are integrated sequentially in one framework to assist the radiologists with a final diagnosis decision. In this paper, we introduce the final step of breast mass classification and diagnosis using a stacked ensemble of residual neural network (ResNet) models (i.e. ResNet50V2, ResNet101V2, and ResNet152V2). The work presents the task of classifying the detected and segmented breast masses into malignant or benign, and diagnosing the Breast Imaging Reporting and Data System (BI-RADS) assessment category with a score from 2 to 6 and the shape as oval, round, lobulated, or irregular. The proposed methodology was evaluated on two publicly available datasets, the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and additionally on a private dataset. Comparative experiments were conducted on the individual models and an average ensemble of models with an XGBoost classifier. Qualitative and quantitative results show that the proposed model achieved better performance for 1) Pathology classification with an accuracy of 95.13%, 99.20%, and 95.88%; 2) BI-RADS category classification with an accuracy of 85.38%, 99%, and 96.08% respectively on CBIS-DDSM, INbreast, and the private dataset; and 3) shape classification with 90.02% on the CBIS-DDSM dataset. Our results demonstrate that our proposed integrated framework could benefit from all automated stages to outperform the latest deep learning methodologies. Mass classification BI-RADS Shape Stacked ensemble Computer-aided diagnosis (CAD) Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 Figure 10 Figure 11 Introduction Over years, breast cancer has remained the most frequently diagnosed non-skin cancer and the leading cause of death among females with a rate of 32% of total cancer cases [ 1 ]. According to the American Cancer Society, it is estimated that over 290,000 new cases will be reported and 43,780 women will die from breast cancer in 2022 [ 2 ]. Early detection and diagnosis of breast cancer is the most effective way to treat this disease and reduce the mortality rate [ 3 ]. Mammography has been proven the most reliable and preferred tool used by radiologists to screen and investigate suspicious breast lesions [ 4 ]. However, with the increase in the number of daily-screened mammograms, an efficient diagnostic methodology is necessary to assist doctors in the timely procedure of breast cancer. Thus, computer-aided diagnosis (CAD) systems, that perform computational image analysis, could provide a second suggestion and reading to the final examination of the experts regarding the presence of breast cancer [ 5 , 6 ]. A completely integrated CAD system would start its first stage, the detection and localization of suspicious lesions and distinguishing between their types, i.e. mass, calcification, architectural distortion, etc. Then, at a second stage, the CAD system should perform a segmentation of the obtained region of interest (ROI) surrounding the breast lesion in order to recognize its anatomical contour and remove its tissue background without losing its shape precision. Finally, diagnostic information can be extracted regarding the lesion’s pathology to classify the decided lesion as either malignant or benign, and identify its characteristics such as tumor grading using Breast Imaging Reporting and Data System (BI-RADS) score, and shape categorization. As the automated procedure relies on connected stages, each output information must be generated precisely to generate a fast and accurate final decision. Therefore, different algorithms have been widely implemented in CAD systems, and the most common used are conventional machine learning classifiers and threshold-based methods that are based on handcrafted features [ 7 , 8 , 9 ]. With the practical challenges that breast tumors offer due to their variation in size, shape, location, and texture, there has been a significant need to improve the overall performance of CAD systems and reduce false positive and negative cases. With the recent progress in computers and their enhanced computational capacity and speed, deep learning methodology has been broadly suggested in biomedical applications and particularly in CAD systems for mammography [ 10 , 11 , 12 ]. In the last two decades, deep learning has shown a growing success in many computer vision tasks and has proven a capability to overcome complex problems in the medical imaging domain. As a result, several works have been suggested and applied particularly in mammography, such as for tumors detection [ 13 , 14 ], breast lesions segmentation [ 15 , 16 ], and classification [ 17 ]. Recently, deep learning models have exceeded the simple adaptation of convolutional neural networks (CNN) algorithms to present several advanced architectures that outperformed the image classification results. In fact, the CNN architecture model was initially suggested for image classification and has been the base of many popular state-of-the-art architectures such as ResNet, AlexNet, EfficientNet, and VGG, etc. Consequently, many works have studied and applied the recent classification models for breast lesions classification, and have been employed in CAD systems in different methodologies such as using ensemble learning [ 18 , 19 ], transfer learning [ 20 , 21 ], and fusion modeling [ 22 , 23 ]. Technical Background Several research studies have attempted to suggest machine learning methods for computer-aided diagnosis (CAD) systems to assist experts in their final diagnostic decisions and have focused on improving results of breast mass classification in digital mammography. In this context, Dhahri et al. [ 24 ] used a Tabu search to select the most significant features and then fed them into a K-Nearest Neighbors (KNN) algorithm to classify breast lesions into malignant or benign. Since their development, many studies have given more attention to incorporating deep learning methods in CAD systems as they showed better efficiency than traditional CAD systems, which require extensive features extraction. For instance, an end-to-end approach was developed by Shen et al. [ 25 ] to classify digital mammograms into cancer or normal. The work presented a modern CNN structure using VGG network and the residual network (ResNet), and achieved an area under the roc curve (AUC) of 0.91 on the CBIS-DDSM dataset and an AUC of 0.98 on the INbreast dataset. Another end-to-end model, called DiaGRAM, was built by Shams et al. [ 26 ] that combined CNN and Generative Adversarial Networks (GAN). The work was conducted to classify mammograms as benign or cancerous and showed an accuracy of 89% on the DDSM dataset and 93.5% on the INbreast dataset. An improved deep learning method, called the DenseNet-II model, was invented in a work by Li et al. [ 27 ] for classification of benign and malignant mammograms. The model was applied on a private collection of mammograms and reached an accuracy of 94.55%. Accordingly, a model for mass classification was proposed by Zhang et al. [ 28 ] that fused texton features with deep CNN features and achieved an accuracy of 94.30% on the CBIS-DDSM dataset. In another work by Muramatsu et al. [ 29 ], a CNN model’s performance was improved by adding synthetic data generated from lung nodules in computed tomography (CT) using cycle GAN. The classification performance was tested on a DDSM dataset and achieved an accuracy of 81.4%. Recently, Chakravarthy et al. [ 30 ] proposed a customized method that integrated deep learning with an extreme learning machine (ELM) for classifying abnormal ROI images into malignant or benign. The proposed work achieved a maximum accuracy of 97.19% on DDSM, 98.13% on the Mammographic Image Analysis Society (MIAS) dataset and 98.26% on INbreast datasets. In a recent work by Khan et al. [ 31 ], multi-view feature fusion (MVFF) based-CAD system was implemented to increase the performance of CNN by combining information of four views of mammograms in order to classify them into malignant or benign with an AUC of 0.84 on the CBIS-DDSM and mini-MIAS databases. Moreover, Kumar et al. [ 32 ] suggested a classification framework for breast density using an ensemble of 4-class neural network classifiers. The work showed a classification accuracy of 90.8% on the DDSM dataset. Besides ensemble learning methodology, transfer learning was also adapted with deep learning techniques to develop an approach for differentiation between benign and malignant breast cancer. Hence, in a work by Alkhaleefah et al. [ 33 ], double-shot transfer learning (DSTL) was used by fine-tuning various pre-trained networks once on an ImageNet dataset, and another time on a larger dataset similar to the target dataset. The method was trained on the CBIS-DDSM and showed a better performance than single-shot transfer learning with an average AUC of 0.99 on the MIAS dataset and 0.94 on the BCDR dataset. Similarly, Falconí et al. [ 34 ] used transfer learning on a NasNet Mobile model and fine tune on VGG models to classify mammogram images according to the BI-RADS scale that achieved an accuracy of 90.9% on the INbreast dataset. Recently, a work by Medeiro et al [ 35 ] combined DenseNet201 and multi-perceptron layer (MLP) models to classify the pathology within BI-RADS levels 3 and 4 for malignancy of breast masses. The model achieved an accuracy of 63% surpassing the performance of a human expert by 9.0%. To accomplish an efficient mass classification and diagnosis procedure, researchers have shown that capturing texture and morphological characteristics could help doctors understand the nature of the breast tumor and assess its malignancy scale. For instance, a research by Bi et al. [ 36 ] showed that the probability of malignancy is highly correlated with the shape and morphology of a breast lesion. Therefore, several works have incorporated the segmentation stage to provide a complete, significant diagnosis. In a previous work by Tsochatzidis et al. [ 37 ] modified convolutional layers of a CNN to integrate both input images and their corresponding segmentation maps in order to improve the diagnosis of breast cancer. The method was applied on DDSM-400 and CBIS-DDSM datasets and achieved a diagnosis performance of AUC of 0.89 and 0.86. Similarly, a dual convolutional neural network was suggested by Li et al. [ 38 ], which computed the mass segmentation and simultaneously predicted the diagnosis results. The model contributed an improvement to the mass segmentation and cancer classification problem at the same time and achieved an AUC of 0.85 on the DDSM dataset and 0.93 and the INbreast dataset. Recently, most of the developed CAD systems have automated the breast cancer diagnosis procedure that gets an entire mammogram image and returns the final diagnosis. Thus, many studies have integrated the first stage of identifying the suspicious region of breast lesions and based on its automated output, performed the segmentation and classification tasks. For instance, Sarkar et al. [ 39 ] proposed an automated CAD system that detects suspicious regions of potential lesions using a deep hierarchical prediction network and then classifies them into mass or non-mass, and finally into malignant or benign using a CNN structure. The work was tested and achieved an accuracy of 98.05% on the DDSM dataset and 98.14% on the INbreast dataset. Another fully automated system by Dhungel et al. [ 40 ] for breast mass classification integrated mass detection and segmentation in a complete CAD system. The methodology used a multi-scale deep belief network (m-DBN) classifier followed by a cascade of CNNs and random forest classifiers for false positive reduction for mass detection, a conditional random field (CRF) for mass segmentation, and a multi-view deep residual neural network (mResNet) for mass classification. The proposed work achieved an AUC of 0.8 on the INbreast dataset. Another recent work by Singh et al. [ 41 ] presented automatic workflow that detects breast tumor regions from mammograms using the Single Shot Detector (SSD), and then outlines its segmented mask using conditional Generative Adversarial Network (cGAN) that was finally used for shape classification using a CNN. The framework achieved an overall accuracy of 80% for the shape classification. Similarly, Al-Antari et al. [ 42 ] proposed a fully integrated CAD system for digital mammograms via deep learning techniques. It started with a mass detection using the You-Only Look Once (YOLO) architecture model, then performed a mass segmentation on the detected regions using Full resolution convolutional network (FrCN), and finally classified the detected and segmented masses into benign or malignant using a CNN model. The entire framework had an overall classification accuracy of 95.64% on the INbreast dataset. The mass classification step was differently solved in a recent work of Al-Antari et al. [ 43 ] that separately adopted three conventional deep learning models including regular feedforward CNN, ResNet-50, and InceptionResNet-V2. The work achieved a maximum accuracy of 95.32% on the INbreast dataset. Inspired by the continuous success of the CNN model and its variations for breast mass classification, we propose a stacked ensemble of residual network (ResNet) models to classify and diagnose previously detected and segmented mass lesions. The proposed model uses three different architectures of the ResNet model, ResNet50V2, ResNet101V2, and ResNet152V2 that are transferred and fine-tuned on our mammography datasets. The models’ layers are stacked together and reconfigured into an entire model for an overall classification and diagnosis of 1) the pathology as malignant or benign; the BI-RADS category as assessment score from 2 to 6; and 3) the lesions’ shape as round, oval, lobulated, or irregular. The presented work will serve as the last stage of an integrated framework for a breast cancer CAD system. The previous stages were proposed in recent works by Baccouche et al. where detection and classification step was first applied using a YOLO-based fusion model to localize and identify suspicious breast lesions as mass or classification [ 23 ], and then using only the detected masses, a Connected-UNets model was suggested for breast mass segmentation and improved with combining real and synthetic data generated by CycleGAN model [ 44 ]. In this paper, we conduct the final stage in a CAD system, breast mass classification and diagnosis, using a stacked ensemble of neural networks models. The proposed methodology completes our recent works conveyed for breast lesions detection and classification from entire mammograms, and followed by a breast mass segmentation step. The work presents an integrated framework for CAD system for breast cancer as the performance relies on three connected stages, and the current step generates the final decision about the breast mass’ pathology (i.e. benign or malignant), its BI-RADS category (i.e. score from 2 to 6), and its shape (i.e. oval, round, lobulated, or irregular). The paper is inspired by ensemble models learning and fusion modeling that showed high efficiency in many recent studies. The suggested methodology was performed on two most popular public mammography datasets: Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and on a private collection of mammograms. Results All experiments for the proposed methodology were conducted using Python 3.6 on a PC with the following specifications: Intel(R) Core (TM) i7-8700K processor with 32 GB RAM, 3.70 GHz frequency, and one NVIDIA GeForce GTX 1090 Ti GPU. Data preparation Mammograms are often collected using a scanning machine of digital X-ray mammography that usually compresses the breast and consequently it degrades the images. Therefore, we apply preprocessing techniques to remove the additional noise and correct the data using a histogram equalization that smooths the pixel distribution. Furthermore, the pre-trained ResNet models require input image size of 224 x 224; therefore, we resize the detected and segmented ROIs from 256 x 256 using an inter-area resampling interpolation. Finally, all images are normalized to a range of [0, 1]. Samples of input data for each classification class are illustrated in Figure.1 where ROIs are distributed according to different class labels from the mammography datasets. Evaluation metrics All classification tasks are evaluated overall using the accuracy, and area under curve (AUC) that reflect the performance of the model while considering the unbalanced mammography datasets. Particularly, for pathology classification, which presents a binary-classes case, we use three additional metrics called sensitivity, specificity scores, and F1-score, as shown in Eq. 1, Eq. 2, and Eq. 3. The F1-score is a coefficient that represents a harmonic average between the specificity and sensitivity, where its maximum score of 1 indicates perfect specificity and sensitivity and of 0 the worst performance. Moreover, the accuracy score is a rate of correct predictions over all cases as detailed in Eq. 4 where TP, TN, FP, and FN are defined per predicted class to represent the number of true positive, true negative, false positive, and false negative predictions, respectively. \(Sensitivity=\frac{TP}{TP+FN}\) (1) \(Specifictity=\frac{TN}{TN+FP}\) (2) \(F1-score=\frac{2\times TP}{2\times TP+FP+FN}\) (3) \(Accuracy=\frac{True prediction}{Total cases}=\frac{TP+TN}{TP+FP+TN+FN}\) (4) In pathology classification, positive refers to the malignant class and negative refers to the benign class. In BI-RADS category and shape classification, macro averaging is used to compute the accuracy and the AUC scores. Consequently, a confusion matrix can be driven from these measurements to show the tradeoff between the true and predicted class labels. Hyperparameters tuning Extensive experiments with different variations in hyperparameters were conducted to select the best parameters for the base ResNetV2 model. Considering their effect on the classification performance, only hyperparameters detailed in Table.1 were tuned to select the best configured network that outperforms the evaluated networks on all mammography datasets. Table.1: Hyperparameters for the ResNetV2 base model Hyperparameters Values explored Description Batch size 32, 64 mini-batch training size Epochs 20, 30, 50 Number of training epochs Dropout 0%, 20%, 30% % of neuron of hidden layers “dropped” for regularization LR 10 − 1 , 10 − 2 , 10 − 3 Learning rate for the Adam optimizer Smoothing 0% 20%, 25% % of label smoothing for the loss function For all datasets, we randomly split images for each class into groups of 80% for training, 20% divided equally between testing and validation sets. In each experiment, the same trainable parameters were used and each hyperparameter was varied accordingly. For all datasets and classification tasks, we used Adam optimized and evaluation was reported with a weighted accuracy score to reflect the class imbalance during the training and testing. The loss function was employed according to the classification task, a Binary Cross-entropy function for binary classes and Categorical Cross-entropy for multiple classes. In both cases, a label smoothing technique for regularization to help overcome overfitting and provide a generalized model. The technique works by explicitly updating the labels during the loss function and decreasing the model’s confidence when it starts diverging [45]. In addition, training was monitored using a method that reduces the learning rate if the accuracy stops improving. Thus, we applied the stated strategy with a factor of 0.5 when the accuracy did not improve after two iterations. Conclusively, the best evaluation was reported with a batch size of 32, 30 epochs, a dropout rate of 30%, a learning rate of 10 − 2 , and a smoothing label of 25%. Quantitative classification results The proposed breast mass classification model was trained and compared to single base models for each presented task on the different mammography datasets. We also compared the stacked ensemble of models to a conventional average of different models’ weights with an XGBoost classifier. Pathology classification As shown in Table.2, Table.3, and Table.4, the pathology classification results are compared between different models respectively for CBIS-DDSM, INbreast, and private datasets. It is reasonable to mention that because the private dataset includes only malignant cases, we trained and tested the model on a combination of all datasets. Table.2: Pathology classification results on the CBIS-DDSM dataset Model Accuracy Sensitivity Specificity F1-score AUC Model1: ResNet50V2 89.97 0.89 0.91 0.9 0.9 Model2: ResNet101V2 93.57 0.92 0.95 0.94 0.93 Model3: ResNet152V2 92.11 0.92 0.92 0.92 0.92 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 91.04 0.85 0.98 0.91 0.91 Stacked Ensemble of models 95.13 0.93 0.97 0.95 0.95 Table.3: Pathology classification results on the INbreast dataset Model Accuracy Sensitivity Specificity F1-score AUC Model1: ResNet50V2 98.52 1.0 0.93 0.97 0.96 Model2: ResNet101V2 95.58 1.0 0.80 0.93 0.9 Model3: ResNet152V2 96.6 1.0 0.90 0.91 0.94 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 97.9 1.0 0.96 0.98 0.97 Stacked Ensemble of models 99.2 1.0 0.98 0.99 0.99 Table.4: Pathology classification results on the Private dataset Model Accuracy Sensitivity Specificity F1-score AUC Model1: ResNet50V2 92.83 0.94 0.90 0.91 0.92 Model2: ResNet101V2 94.18 0.94 0.95 0.93 0.94 Model3: ResNet152V2 94.60 0.94 0.94 0.94 0.94 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 94.89 0.94 0.97 0.93 0.94 Stacked Ensemble of models 95.88 0.93 0.97 0.95 0.96 The comparative results show that the proposed stacked ensemble of models performs better than the base ResNet models having different numbers of deep layers (i.e. ResNet50V2, ResNet101V2 and ResNet152V2). Accordingly, our proposed methodology outperformed the average ensemble of models with an XGBoost classifier that performed slightly better than individual models. We clearly notice a high accuracy of 95.13% on the CBIS-DDSM dataset, 99.2% on the INbreast dataset, and 95.88% on the private dataset. Besides, our proposed model achieved a high sensitivity rate of 0.93 on the CBIS-DDSM dataset, 1.0 on the INbreast dataset, and 0.93 on the private dataset. Consequently, the results emphasize generally the advantage of ensemble learning technique on improving the classification performance, and particularly the improvement achieved by the stacking method using deep learning models. Moreover, the pathology classification performance was compared against the different models using the AUC over the test sets of all datasets. Figure.2 shows plots of the Receiver Operating Characteristic (ROC) curves of the True positive Rate (TPR) against the False Positive Rate (FPR), and we clearly notice that the proposed model outperformed all experimental techniques with an AUC of 0.95 for the CBIS-DDSM dataset, 0.99 for the INbreast dataset, and 0.96 for the private dataset. BI-RADS category classification Results shown in Table.5, Table.6 and Table.7 for BI-RADS category classification illustrate the comparison between different models for all mammography datasets. As mentioned in the section on Datasets description, each dataset has different class labels that vary from category 2 to category 6. Table.5: BI-RADS category classification results on the CBIS-DDSM dataset Model Accuracy AUC Model1: ResNet50V2 80.07 0.93 Model2: ResNet101V2 77.12 0.92 Model3: ResNet152V2 80.08 0.93 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 79.48 0.85 Stacked Ensemble of models 83.84 0.94 Table.6: BI-RADS category classification results on the INbreast dataset Model Accuracy AUC Model1: ResNet50V2 98.0 0.94 Model2: ResNet101V2 98.0 0.97 Model3: ResNet152V2 96.1 0.92 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 97.1 0.99 Stacked Ensemble of models 99.0 1.00 Table.7: BI-RADS category classification results on the Private dataset Model Accuracy AUC Model1: ResNet50V2 91.91 0.91 Model2: ResNet101V2 92.43 0.92 Model3: ResNet152V2 94.25 0.94 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 92.95 0.93 Stacked Ensemble of models 96.08 0.95 The classification results presented above demonstrate a clear improvement of the performance using our proposed stacked ensemble of models comparing to the basic models with an accuracy of at least 3.78% on the CBIS-DDSM dataset, 1% on the INbreast dataset, and 1.83% on the private dataset. Moreover, our methodology achieved a better AUC score than the average ensemble model with an XGBoost classifier where we notice a high AUC of 0.94 for the CBIS-DDSM dataset, 1.00 on the INbreast dataset, and 0.95% on the private dataset. This can be confirmed with a visual comparison of ROC curve plots between employed models as illustrated in Figure.3. Shape classification Lastly, the proposed model was trained on the CBIS-DDSM dataset for classifying the shape of breast masses, as it is the only dataset that possesses shape annotation by experts. Equivalently, all trained models were tested, and comparison is shown in Table.8. Undoubtedly, our suggested stacked ensemble of models had the highest accuracy score of 90.02% among the employed models, which improved the performance of separate models notably with 1.7% and remarkably with 10.66% compared to the average ensemble of models with an XGBoost classifier. Table.8: Shape classification results on the CBIS-DDSM dataset Model Accuracy AUC Model1: ResNet50V2 75.51 0.90 Model2: ResNet101V2 89.90 0.95 Model3: ResNet152V2 88.32 0.97 Average Weights of Model1, Model2 and Model3 + XGBoost Classifier 79.36 0.84 Stacked Ensemble of models 90.02 0.98 Furthermore, Figure.4 presents a comparison of ROC curve plots for the different employed models and represents the AUC score accordingly. We clearly notice that our proposed model had the highest AUC of 0.98 among the presented models, which was close to the ResNet152V2 performance but with a slightly better accuracy rate. Qualitative classification results Previous comparison results highlighted that our proposed methodology yielded the best results for the different classification tasks. Consequently, we analyzed the classification prediction between different datasets using the confusion matrix that summarizes the results across the class labels. Figure.5, Figure.6 and Figure.7 respectively present the normalized confusion matrix plots for each classification problem. As indicated below, the INbreast dataset had the best pathology classification tradeoff between malignant and benign classes, and this can be explained with the high-quality resolution of the mammograms collected in FFDM format that helps distinguish between the two class labels. The private dataset had also a remarkable confusion matrix with close recall and precision scores, which were similar to the CBIS-DDSM dataset’s performance. Moreover, the INbreast dataset had the best BI-RADS categorization tradeoff with a notable prediction per class from 0.92 to 1.0. Concerning the private dataset, it has only two BI-RADS categories 4 and 5, and we notice a similar satisfying confusion matrix with prediction scores of 0.93 and 0.96. The CBIS-DDSM dataset has a slightly worse tradeoff for the BI-RADS category classification and this is due to the low resolution of deteriorated ROI images from the digitized X-rays mammograms. In fact, the confusion matrix shows values from 0.80 to 0.89 and we clearly notice that the four categories have a close prediction score due to the similarity of the pixel distribution caused by the quality presented in the public dataset. Finally, the confusion matrix for the shape classification showed an overall sufficient tradeoff between the class labels. We clearly observe similar predicted results for the irregular and lobulated cases with a maximum value of 0.93, and this can be interpreted by the close appearance of the two lesions’ shape. However, oval and round cases had worse results, and in particular, the round class label had a performance score of 0.73. Additionally, we reported the classification results of the final integrated CAD system using the segmentation step and compared it to results of the proposed method without using the segmented ROIs. Table.9 shows a better classification performance for each task on all mammography datasets, where the pathology classification demonstrated improvement of 4.26% on the CBIS-DDSM dataset, 4% on the INbreast dataset, and 5.5% on the private dataset. Accordingly, the BI-RADS category classification presented enhanced performance using the detected and segmented images with accuracy difference of 3.69% on the CBIS-DDSM dataset, 1.5% on the INbreast dataset, and 0.38% on the private dataset. Finally, the CBIS-DDSM dataset had improved the shape classification results with 4.36% accuracy. Consequently, it is clearly observed that the CAD system with the integrated detection and segmentation stages achieved much better results for the classification and diagnosis of breast masses. Table.9: Comparison of classification performance (accuracy %) using the proposed CAD system with and without mass segmentation Dataset Pathology BI-RADS Category Shape CAD system without mass segmentation CAD system with mass segmentation CAD system without mass segmentation CAD system with mass segmentation CAD system without mass segmentation CAD system with mass segmentation CBIS-DDSM 90.87 95.13 81.69 85.38 85.66 90.02 INbreast 95.20 99.20 97.50 99 NA NA Private 90.38 95.88 95.70 96.08 NA NA Finally, a comparison of results of the latest state-of-the-art methods and similar models to classify the breast masses is listed in Table.10. Our proposed CAD system that integrates the previous detection and segmentation steps and the current proposed classification framework outperformed the previous deep learning models applied for pathology, BI-RADS category, and shape classification. Compared with other techniques that used segmented ROIs, we exceeded the performance of the work by Falconí et al. [20] that only achieved an accuracy of 78.4% using the MobileNet model on the CBIS-DDSM dataset. On the other hand, we also outperformed the work of Alkhaleefah et al. [33] even though they did not use segmented input images and reported an accuracy of 93.47%. Moreover, recent works on the INbreast dataset were all surpassed where the highest accuracy of 98.26% was reported by Chakravarthy et al. [30] using the ICS-ELM algorithm on original ROI masses. We also reported better accuracy score for the pathology classification applied on segmented ROIs from the INbreast dataset, where Al-Antari et al. [42] only achieved an accuracy of 95.64% and an AUC score of 0.94. The results of the BI-RADS categorization also outperformed the previous works on CBIS-DDSM dataset with the work suggested by Medeiros et al. [35] that applied DenseNet201 on original ROI masses and only achieved an accuracy of 63.4%. No previous paper applied the BI-RADS category classification on segmented images and therefore we could not compare with our proposed work. Accordingly, our method surpassed the performance on the INbreast dataset by the work of Falconí et al. [34] that only reported an accuracy of 90.9% using NasNet and VGG models. Lastly, our methodology gained the best shape classification performance compared to a recent work of Singh et al. [41] that applied a CNN model on a similar dataset, and it is reasonable to say that this reviewed work is the only comparable work that applied shape classification on detected and segmented ROIs but only achieved an accuracy of 80%. Table.10: Comparison of the proposed methodology and state-of-the-art methods Reference Year Method Classification Dataset Images Segmented Accuracy AUC [40] 2017 multi-view deep residual neural network (mResNet) Pathology INbreast Yes -- 0.80 [42] 2018 Ensemble of AlexNet-based CNN Pathology INbreast Yes 95.64 0.94 [20] 2019 MobileNet and NasNet + fine tuning Pathology CBIS-DDSM Yes 78.4 -- [21] 2020 VGG16 + fine tuning Pathology CBIS-DDSM No 84.4 0.84 [30] 2020 Improved Crow-Search Optimized Extreme Learning Machine (ICS-ELM) algorithm Pathology INbreast No 98.26 -- [33] 2020 AlexNet, VGG, GoogLeNet, ResNet + fine tuning Pathology CBIS-DDSM No 93.47 0.97 [35] 2020 DenseNet201 + MLP BI-RADS CBIS-DDSM No 63.4 -- [34] 2020 NasNet + fine tuning on VGG16 and VGG19 BI-RADS INbreast No 90.9 0.99 [41] 2020 CNN Shape DDSM Yes 80 0.80 [38] 2021 DualCoreNet: Texture and shape features fusion Pathology INbreast Yes -- 0.93 Proposed CAD 2022 Stacked Ensemble of ResNet models Pathology CBIS-DDSM Yes 95.13 0.95 INbreast 99.20 0.99 Private 95.88 0.95 BI-RADS CBIS-DDSM Yes 85.38 0.94 INbreast 99 1.00 Private 96.08 0.95 Shape CBIS-DDSM Yes 90.02 0.98 Discussion Deep learning models have recently revealed remarkable success in breast mass classification and diagnosis for many CAD systems. The CNN architecture model was mostly modified on many proposed studies and combined with other recent techniques such as transfer learning and ensemble models learning for a better classification performance. In this study, we have implemented a stacked ensemble of ResNet models to classify breast masses as malignant or benign and diagnose their BI-RADS category assessment with a score from 2 to 6 and their shape as oval, round, lobulated or irregular. The results of the proposed methodology showed the classification performance’s improvement compared to the individual architectures and to the other methods applied on the existing benchmark datasets. Table.10 shows that we achieved the highest pathology classification performance on the two public datasets: CBIS-DDSM with an accuracy of 95.13% and an AUC score of 0.95, and INbreast with an accuracy of 99.20 and an AUC score of 0.99. Furthermore, we surpassed the results of other models for the BI-RADS categorization on the CBIS-DDSM dataset with an accuracy of 85.38% and an AUC score of 0.94, and on the INbreast dataset with an accuracy of 99% and an AUC score of 1.0. We also reported the highest results on the shape classification for the CBIS-DDSM dataset with an accuracy of 90.02% and an AUC score of 0.98. Comparing with the similar frameworks that applied the presented classification tasks on segmented ROI masses, our model outperformed the MobileNet and NasNet models [ 21 ] for the pathology classification on the CBIS-DDSM dataset and the Ensemble of AlexNet-based CNN model [ 46 ] on the INbreast dataset. Moreover, the shape classification achieved better results on a similar dataset DDSM that was evaluated with an individual CNN model [ 47 ]. As a result, the stacking model technique provided an efficient way to learn from various depths of neural networks and combine them in another neural network classifier model to benefit from the different weights that were trained individually. The work integrated our recent works of the YOLO-based fusion models [ 23 ] and the Connected-UNets model [ 44 ] that generated the detected and segmented ROIs of breast masses. Indeed, an increase of performance using the segmented ROIs, as shown in Table.9, indicates the advantage of masking the background tissues from the tumors’ boundaries to help improve the overall classification and diagnosis, and decrease the false positive and negative rates. Limitations of the proposed methodology can occur on the long training time of 0.74s per epoch, which is due to the high number of trainable parameters and computations of the ResNetV2 model. In conclusion, this work presents the final stage of an integrated framework for a breast cancer CAD system via deep learning models. The three stages of detection, segmentation and classification aim to provide a complete clinical tool that can assist radiologists with a second suggestion with an automated mass tumor diagnosis. Future works can include combining different mammography datasets and improving the long training of deep learning models for the classification task. Methods In this study, we propose a stacked ensemble of models to classify and diagnose detected and segmented breast masses. The base model comes from the ResNet architecture and its variations. Our methodology employs different strategies: transfer learning, stacked ensemble learning, and image data augmentation. ResNet base model: transfer learning and fine-tuning Since its introduction, ResNet is a deep CNN architecture suggested by He et al. [46] that has been one of the recent architectures that has known common success in medical imaging applications [47, 48]. ResNet uses residual blocks with skip connection between layers to bypass a few convolution layers at a time. This architecture accelerated the convergence of a larger number of deep layers, and consequently it has been found efficient to provide a compact representation of input images and improve the classification task performance [22]. The ResNet has some common architectures such as ResNet-50, 101, and 152, [37] which indicate the number of deep layers. Alternatively, ResNet architecture presented an improved version ResNetV2 by He et al. [49], where the last ReLU was removed to clear the shortcut path using a simple identity connection as shown in Supplementary Figure.1. Our methodology employs three pre-trained ResNetV2 architectures, detailed below in Supplementary Table.1. Training a deep learning model often requires a large amount of annotated data that helps optimizing the high number of parameters and computations needed in the architecture. However, a limited size of medical imaging datasets are usually available that suffer from either missing labels or imbalanced data distribution. To overcome these challenges, transfer learning has been a common solution used in many recent medical image application [50, 51] by training a model on a large and diverse dataset (i.e. ImageNet, MSCOCO, etc.) to capture universal features like curves, edges, and boundaries in its early layers that are relevant for image classification. After that, the pre-trained model should be alerted and fine-tuned on a custom and specific dataset to reflect the final classification. This procedure provides a fast and generalizable training of small datasets and avoids the overfitting problem that deep learning commonly suffers from. As Figure.8 indicates, we apply transfer learning to the base architecture ResNetV2 for our proposed methodology to become a TF-ResNetV2. In fact, the model was initially pre-trained on ImageNet, and then the first four residual blocks of layers were frozen with exception to the BN layers that needed to be retrained in order to improve the training convergence. After that, the entire architecture was modified by adding another FC layer with size of 1024, followed by a dropout regularization layer to maintain a generalization aspect for the training. A new final FC layer was placed according to the number of classes for each classification task and the entire TF-ResNetV2 is re-trained. Stacked Ensemble of ResNet models for Breast mass classification Ensemble learning has been considered efficient to improve the classification task results. Combining weaker classifiers to create a better final classification prediction has been adopted by either bagging, boosting, or stacking models. While bagging is achieved by learning independently from different models and then averaging the predictions, boosting happens by sequentially learning from homogenous learners and iteratively combining them into a final model. On the other hand, stacking has been considered a way to learn different weak learners in parallel and combine them into a meta-model that is later trained to achieve the classification prediction [52]. We propose a stacked ensemble of three different ResNet models to conduct our classification tasks. After removing the last FC layer of each ResNetV2 architecture, a two-layers network is considered as a meta-classifier model that concatenates the three models’ layers, and stacks three different FC layers of size 1000, 100 and 10, coupled with activation functions Sigmoid and ReLU. As shown in Figure.9, after training independently ResNet50V2, ResNet101V2 and ResNet152V2, pre-trained weights of each model were extracted as images features of size 1024 based on previous layer predictions and considered as new input of the entire stacked ensemble of ResNet models for the final class prediction. Integrated framework: Mass detection, segmentation and classification Our final framework should now be complete with all automated steps for breast cancer analysis and diagnosis. Therefore, as shown in Figure.10, the integrated framework detects and localizes breast masses in a first step using YOLO-based fusion models [23], which only requires an entire mammogram image and outputs bounding boxes around specious lesions. The model was evaluated and provided a maximum detection accuracy of 98.1% for mass lesions. The next step should segment the detected ROI of breast masses and generate a binary mask image where only the boundary of the lesions is visible. The second step is achieved using the proposed Connected-UNets model [44] that was improved by synthetic data, which is generated by CycleGAN. The segmentation step was conducted on ROI images scaled to an optimal size of 256 x 256 pixels. Evaluation showed a high Dice score of 95.88% and Intersection over Union (IoU) of 92.27%. After that, the segmented and detected ROI of breast masses generated with masked tissue are used for the third and final classification step. The stacked ensemble of ResNet models is trained independently on the input ROI masses for each classification task to finally predict the pathology as either malignant or benign, the BI-RADS category with an assessment score between 2 to 6, and the shape as either round, oval, lobulated or irregular. Datasets Similar to our previous works, we evaluate the proposed classification methodology on two public datasets, the CBIS-DDSM and INbreast datasets, and on an independent private dataset. The CBIS-DDSM dataset [53] is an updated and standardized version of the Digital Database for Screening Mammography (DDSM), where images were reviewed by radiologists to eliminate inaccurate cases and converted from Lossless Joint Photographic Experts Group (LJPEG). It contains 2907 mammograms from 1555 unique patients, where 1467 are mammograms with mass lesions acquired with two different views (i.e. MLO and CC). Original mammograms have an average size of 3000 x 4800 pixels and are associated with their pixel-level annotation and class labels (i.e. Pathology, BI-RADS category and Shape). The INbreast dataset [54] is a public database of Full-Field Digital Mammography (FFDM) images in DICOM format. It contains 410 mammograms from 115 unique patients where only 107 cases present mass lesions in both MLO and CC views. Original mammograms have an average size of 3328 x 4084 pixels and include pixel-level annotation and class labels (i.e. Pathology and BI-RADS category). The private dataset is a collection of mammograms from the National Institute of Cancerology (INCAN) in Mexico City and contains stage 3 and 4 of breast cancer with 389 cases from 208 unique patients having mass lesions. Images have an average of 300 x 700 pixels acquired from different views (i.e. CC, MLO, ML and AT), and include associated pixel-level annotation and class labels (i.e. Pathology and BI-RADS category). Figure.11 illustrates samples of original mammograms and their ROI masses compared to the detected and segmented ROI masses from different datasets. As the datasets were explored continuously during the previous studies, the original mammograms that included mass and calcification cases were used during the first step of detection and localization; therefore detected ROIs of only mass cases were retained for the second step of segmentation. It is fair to mention that some mammograms have multiple ROIs and hence the number of detected and segmented ROI masses used for the third step of classification and diagnosis may vary. Due to the limited amount of ROI masses in each dataset, raw ROIs data was augmented four times by rotating them with the angles Δθ = {0°, 90°, 180°, 270°}, and transformed twice differently using the Contrast Limited Adaptive Histogram Equalization (CLAHE) method. Table.11 details the data distribution of each mammography dataset regardless of the class labels. Table.11: General Datasets distribution Dataset Raw MGs Data Raw ROIs Data Augmented Data (ROIs*6) CBIS-DDSM 1,467 1,467 8,802 INbreast 107 112 672 Private 389 638 3,828 Moreover, each mammography dataset has a different quality of images in terms of pixel quality, existing annotated labels and class distribution, as detailed in Table.12, 13 and 14. In fact, only the CBIS-DDSM dataset includes true class labels for lesions’ shape. Accordingly, the INbreast dataset indicates cases with BI-RADS score from 2 to 6, however the CBIS-DDSM dataset presents cases in BI-RADS category 2 to 5, and the private dataset has only malignant cases as it acquired breast cancer cases from only stage 3 and 4. Consequently, all mammograms from the private dataset fall into BI-RADS categories 4 and 5. Table.12: Pathology class labels distribution Dataset Pathology Benign Malignant CBIS-DDSM* 4,500 4,302 INbreast** 150 522 Private 0 3830 * Cases with Benign_without_callback are considered Benign ** Cases with BI-RADS score > 3, are considered Malignant otherwise Benign Table.13: Shape class labels distribution Dataset Shape Round Oval Lobulated Irregular CBIS-DDSM 804 2,040 2,112 3,846 INbreast NA NA NA NA Private NA NA NA NA Table.14: BI-RADS category class labels distribution Dataset BI-RADS Category 2 Category 3 Category 4 Category 5 Category 6 CBIS-DDSM 792 1,938 2,328 3,402 0 INbreast 144 78 126 276 48 Private 0 0 1,627 2,195 0 Declarations DATA AVAILABILITY The public mammography dataset CBIS-DDSM generated and analyzed during the current study is available in the Cancer Imaging Archive, https://wiki.cancerimagingarchive.net/display/Public/CBIS-DDSM. The public mammography dataset INbreast generated and analyzed during the current study is available from the corresponding author Inês Domingues, Porto, Portugal, on reasonable request after signing a transfer agreement. The private mammography dataset generated during and analyzed during the current study is available from the corresponding author Cristian Castillo Olea through the oncologist Dr. Eric Ortiz in the National Institute of Cancerology, Mexico. CODE AVAILABILITY The code for custom algorithms and data preprocessing is provided as part of the replication package. It was written in Python v3.6. It is publicly available as a git repository on GitHub at https://github.com/AsmaBaccouche/Stacked-Ensemble-of-Residual-Neural-Networks ACKNOWLEDGEMENTS The authors would especially like to express their gratitude to the National Institute of Cancerology (INCAN) in Mexico City and particularly to Christian Castillo for providing the private mammography dataset. Thanks also to the radiologists Dr. Kictzia Yigal Larios and Dr. Raquel Balbás at FUCAM A. C., and Dr. Guillermo Peralta and Dr. Néstor Piña at Cancer Center Tec100 by MRC International. AUTHOR CONTRIBUTIONS A.B conceived the idea, developed and implemented the methods. B.G-Z helped with formulating and validating the experiments and analysis. B.G-Z and A.S.E supervised the project. A.B wrote the paper. All authors reviewed and edited the manuscript. COMPETING INTERESTS The authors declare no competing interests. ADDITIONAL INFORMATION Supplementary information The online version contains supplementary material available Supplementary Information.pdf. References Ferlay, J. et al. Cancer statistics for the year 2020: An overview. International Journal of Cancer 149 , 778-789 (2021). American Cancer Society. Cancer Statistics Center. http://cancerstatisticscenter.cancer.org. Accessed January 17, 2022. Duffy, S. W. et al. Mammography screening reduces rates of advanced and fatal breast cancers: Results in 549,091 women. Cancer 126 , 2971-2979 (2020). Dibden, A., Offman, J., Duffy, S. W., & Gabe, R. Worldwide review and meta-analysis of cohort studies measuring the effect of mammography screening programmes on incidence-based breast cancer mortality. Cancers 12 , 976 (2020). Rahman, M. M. el al. Machine learning based computer aided diagnosis of breast cancer utilizing anthropometric and clinical features. Irbm 42 , 215-226 (2021). Ramadan, S. Z. Methods used in computer-aided diagnosis for breast cancer detection using mammograms: a review. Journal of healthcare engineering (2020). Yassin, N. I., Omran, S., El Houby, E. M., & Allam, H. Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review. Computer methods and programs in biomedicine 156 , 25-45 (2018). Paramkusham, S., Thotempuddi, J., & Rayudu, M. S. Breast masses classification using contour shape descriptors based on Beam Angle Statistics. In 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA) , 1340-1345. (IEEE, 2021). Li, H., Meng, X., Wang, T., Tang, Y., & Yin, Y. Breast masses in mammography classification with local contour features. Biomedical engineering online 16 , 1-12 (2017). Al-Masni, M. A. et al. Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. Computer methods and programs in biomedicine 157 , 85–94 (2018). Eltrass, A. S., & Salama, M. S. Fully automated scheme for computer‐aided detection and breast cancer diagnosis using digitised mammograms. IET Image Processing 14 , 495-505 (2020). Siddiqui, S. Y. et al. Intelligent breast cancer prediction empowered with fusion and deep learning. Computer, Materials and Continua 67 , 1033–1049 (2021). Aly, G. H., Marey, M., El-Sayed, S. A., & Tolba, M. F. YOLO Based Breast Masses Detection and Classification in Full-Field Digital Mammograms. Computer Methods and Programs in Biomedicine 200 , 105823 (2021). Al-Antari, M. A., Han, S. M., & Kim, T. S. Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms. Computer methods and programs in biomedicine 196 , 105584 (2020). Abdelhafiz, D., Bi, J., Ammar, R., Yang, C. & Nabavi, S. Convolutional neural network for automated mass segmentation in mammography. BMC Bioinformatics 21 , 1–19 (2020). Taghanaki, S. A. et al. Deep semantic segmentation of natural and medical images: a review. Artif. Intell. Rev. 54 , 137–178 (2021). Jiao, Z., Gao, X., Wang, Y., & Li, J. A parasitic metric learning net for breast mass classification based on mammography. Pattern Recognition 75 , 292-301 (2018). Rasti, R., Teshnehlab, M., & Phung, S. L. Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. Pattern Recognition 72 , 381-390 (2017). Rampun, A., Scotney, B. W., Morrow, P. J., & Wang, H. Breast mass classification in mammograms using ensemble convolutional neural networks. In 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom) , 1-6 (IEEE 2018). Falconí, L. G., Pérez, M., & Aguilar, W. G. Transfer learning in breast mammogram abnormalities classification with mobilenet and nasnet. In 2019 International Conference on Systems, Signals and Image Processing (IWSSIP) , 109-114 (IEEE, 2019). Falconi, L. G., Perez, M., Aguilar, W. G., & Conci, A. Transfer learning and fine tuning in breast mammogram abnormalities classification on CBIS-DDSM database. Advances in Science, Technology and Engineering Systems 5 , 154-165 (2020). Ragab, D. A., Attallah, O., Sharkas, M., Ren, J., & Marshall, S. A framework for breast cancer classification using multi-DCNNs. Computers in Biology and Medicine 131 , 104245 (2021). Baccouche, A., Garcia-Zapirain, B., Castillo Olea, C. & Elmaghraby, A. S. Breast lesions detection and classification via yolo-based fusion models. Computer. Material. Contina 69 , 1407–1425 (2021). Dhahri, H., Rahmany, I., Mahmood, A., Al Maghayreh, E., & Elkilani, W. Tabu search and machine-learning classification of benign and malignant proliferative breast lesions. BioMed research international (2020). Shen, L. et al. Deep learning to improve breast cancer detection on screening mammography. Scientific reports 9 , 1-12 (2019). Shams, S. et al. Deep generative breast cancer screening and diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention , 859-867 (Springer, Cham 2018). Li, H., Zhuang, S., Li, D. A., Zhao, J., & Ma, Y. Benign and malignant classification of mammogram images based on deep learning. Biomedical Signal Processing and Control 51 , 347-354 (2019). Zhang, Q. et al. A novel algorithm for breast mass classification in digital mammography based on feature fusion. Journal of Healthcare Engineering (2020). Muramatsu, C. et al. Improving breast mass classification by shared data with domain transformation using a generative adversarial network. Computers in biology and medicine 119 , 103698 (2020). Chakravarthy, S. S., & Rajaguru, H. Automatic Detection and Classification of Mammograms Using Improved Extreme Learning Machine with Deep Learning. IRBM (2021). Khan, H. N., Shahid, A. R., Raza, B., Dar, A. H., & Alquhayz, H. Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. IEEE Access 7 , 165724-165733 (2019). Kumar, I., Bhadauria, H. S., Virmani, J., & Thakur, S. A classification framework for prediction of breast density using an ensemble of neural network classifiers. Biocybernetics and Biomedical Engineering 37 , 217-228 (2017). Alkhaleefah, M. et al. Double-shot transfer learning for breast cancer classification from X-ray images. Applied Sciences 10 , 3999 (2020). Falconí, L., Pérez, M., Aguilar, W., & Conci, A. Transfer Learning and Fine Tuning in Mammogram BI-RADS Classification. In 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) , 475-480 (IEEE, 2020). Medeiros, A., Ohata, E. F., Silva, F. H., Rego, P. A., & Reboucas Filho, P. P. An approach to BI-RADS uncertainty levels classification via deep learning with transfer learning technique. In 2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS) , 603-608 (IEEE, 2020). Bi, W. L. et al. Artificial intelligence in cancer imaging: clinical challenges and applications. CA: a cancer journal for clinicians 69 , 127-157 (2019). Tsochatzidis, L., Koutla, P., Costaridou, L., & Pratikakis, I. Integrating segmentation information into CNN for breast cancer diagnosis of mammographic masses. Computer Methods and Programs in Biomedicine 200 , 105913 (2021). Li, H., Chen, D., Nailon, W. H., Davies, M. E., & Laurenson, D. Dual Convolutional Neural Networks for Breast Mass Segmentation and Diagnosis in Mammography. Preprint at https://arxiv.org/abs/ 2008.02957 (2020). Sarkar, P. R., Prabhakar, P., Mishra, D., & Subrahmanyam, G. Towards automated breast mass classification using deep learning framework. In 2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA) , 453-462 (IEEE, 2019). Dhungel, N., Carneiro, G., & Bradley, A. P. Fully automated classification of mammograms using deep residual neural networks. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) , 310-314 (IEEE, 2017). Singh, V. K. et al. Breast tumor segmentation and shape classification in mammograms using generative adversarial and convolutional neural network. Expert Systems with Applications 139 , 112855 (2020). Al-Antari, M. A., Al-Masni, M. A., Choi, M. T., Han, S. M., & Kim, T. S. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. International journal of medical informatics 117 , 44-54 (2018). Al-Antari, M. A., Al-Masni, M. A., & Kim, T. S. Deep learning computer-aided diagnosis for breast lesion in digital mammogram. Deep Learning in Medical Image Analysis , 59-72 (2020). Baccouche, A., Garcia-Zapirain, B., Castillo Olea, C., & Elmaghraby, A. S. Connected-UNets: a deep learning architecture for breast mass segmentation. NPJ Breast Cancer 7 , 1-12 (2021). Müller, R., Kornblith, S., & Hinton, G. When does label smoothing help?. Preprint at https://arxiv.org/abs/ 1906.02629 (2019). He, K., Zhang, X., Ren, S., & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , 770-778 (2016) Garcia-Gasulla, D. et al. On the behavior of convolutional nets for feature extraction. Journal of Artificial Intelligence Research 61 , 563-592 (2018). Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. A. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence (2017). Yu, X. et al. ResNet-SCDA-50 for breast abnormality classification. IEEE/ACM transactions on computational biology and bioinformatics 18 , 94-102 (2020). He, K., Zhang, X., Ren, S., & Sun, J. Identity mappings in deep residual networks. In European conference on computer vision , 630-645 (Springer, Cham 2016). Chen, Y. et al. Fine-tuning ResNet for breast cancer classification from mammography. In The International Conference on Healthcare Science and Engineering , 83-96 (Springer, Singapore 2018). Bellmann, P., Thiam, P., & Schwenker, F. Multi-classifier-systems: architectures, algorithms and applications. In Computational Intelligence for Pattern Recognition , 83-113 (Springer, Cham 2018). Lee, R. S. et al. A curated mammography data set for use in computer-aided detection and diagnosis research. Scientific data 4 , 1-9 (2017). Moreira, I. C. et al. Inbreast: toward a full-field digital mammographic database. Academic radiology 19 , 236-248 (2012). Additional Declarations No competing interests reported. Supplementary Files SupplementaryInformation.pdf Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Major revision 30 May, 2022 Reviews received at journal 23 May, 2022 Reviewers agreed at journal 19 May, 2022 Reviewers invited by journal 19 May, 2022 Editor assigned by journal 01 Mar, 2022 Editor invited by journal 26 Feb, 2022 Submission checks completed at journal 26 Feb, 2022 First submitted to journal 23 Feb, 2022 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-1389924","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":86710156,"identity":"b50cab82-3b4e-438f-a898-fdc8c368671a","order_by":0,"name":"Asma Baccouche","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABH0lEQVRIiWNgGAWjYLACxgYIzczAYCMD4RpYyIBEJIjQksYD1SLBQ6yWwzxQMdxa+PtPJ35g3GEnz9/eY/i5oOI8D79EcvNnngIJHn4G5oO3eTC1SNzI3SzBeCbZcMaZM8bSM87c5pGckdgmzQN0mGQDW7I1Fi0MN3g3SDC2MScYSOSYMfO23eYxuJHYxjgDqMXgAI+ZNBYt8ufPbv7B2FYP1fLvHEhL80eQFvsD/N+waTE4kLsNaMthqJaGAyAtDRIfQLYw8LBh02J4I3ebReKZ40C/HCuW5jmWzCPZ87ANrEXiMJux5RxMLXJAh934uKMaGGLNGz/z1NjJ8bOnP/6Q8MdGDijy8MYbLN4HgQTswsw4lI+CUTAKRsEoIAgAyZ5c/nZxOJQAAAAASUVORK5CYII=","orcid":"","institution":"University of Louisville","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Asma","middleName":"","lastName":"Baccouche","suffix":""},{"id":86710157,"identity":"4b4f1904-60b7-404d-88bc-b2952b5422f7","order_by":1,"name":"Begonya Garcia-Zapirain","email":"","orcid":"","institution":"University of Deusto","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Begonya","middleName":"","lastName":"Garcia-Zapirain","suffix":""},{"id":86710160,"identity":"7af833c1-9fbd-4ad5-993f-d1e392c47722","order_by":2,"name":"Adel Elmaghraby","email":"","orcid":"","institution":"University of Louisville","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Adel","middleName":"","lastName":"Elmaghraby","suffix":""}],"badges":[],"createdAt":"2022-02-23 18:44:13","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-1389924/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-1389924/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":18753995,"identity":"cf508b1f-07c2-45dc-9c81-aa5f530cf2fb","added_by":"auto","created_at":"2022-03-01 21:37:57","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":313906,"visible":true,"origin":"","legend":"\u003cp\u003eSamples of detected and segmented ROI masses for each class within different classification tasks from different mammography datasets\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/5a63d77ca9da5a06c2c32ff5.png"},{"id":18753505,"identity":"f683debf-5875-49a3-95d4-db100c7887da","added_by":"auto","created_at":"2022-03-01 21:34:57","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":331267,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance of pathology classification using different models in terms of ROC curves and AUC score\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/5684681f776b6d9210af8dc9.png"},{"id":18753512,"identity":"21412a44-eaa6-4880-b475-08627525153d","added_by":"auto","created_at":"2022-03-01 21:34:58","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":329163,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance of BI-RADS classification using different models in terms of ROC curves and AUC score\u003c/p\u003e","description":"","filename":"floatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/e9f85c02d534be87489c1f3a.png"},{"id":18753997,"identity":"d5bf5cd7-112e-4491-a62b-06164ccbe0d2","added_by":"auto","created_at":"2022-03-01 21:37:58","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":35519,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance of Shape classification using different models in terms of ROC curves and AUC score\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/6132736eb3b63e7f693b1667.png"},{"id":18753508,"identity":"03e6426a-3d3c-4ead-8a2d-e7dd8a0025a2","added_by":"auto","created_at":"2022-03-01 21:34:57","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":115422,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix of the stacked ensemble of models for the pathology classification on the mammography datasets\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/887cb787510dcfd2812259f2.png"},{"id":18754490,"identity":"e4835edf-209f-496e-b366-8af0b1990514","added_by":"auto","created_at":"2022-03-01 21:40:57","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":158546,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix of the stacked ensemble of models for the BI-RADS category classification on the mammography datasets\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/ad34b4876ec2b2445d2eabb0.png"},{"id":18754002,"identity":"70956ac1-7465-4b83-86a8-8a95c95fb12e","added_by":"auto","created_at":"2022-03-01 21:37:58","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":86942,"visible":true,"origin":"","legend":"\u003cp\u003eConfusion matrix of the stacked ensemble of models for the Shape classification on the mammography datasets\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/b069991cc40a3a4ae3efd17c.png"},{"id":18754000,"identity":"bad51992-685a-4c92-bcf8-aff3c30ff8ad","added_by":"auto","created_at":"2022-03-01 21:37:58","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":102939,"visible":true,"origin":"","legend":"\u003cp\u003eFramework of the classification base model: a TF-ResNetV2 model which is a ResNetV2 model pre-trained on ImageNet data and modified and fine-tuned on mammography dataset\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/f342bca62097aab506c9b0d3.png"},{"id":18754642,"identity":"d5e24505-d649-458c-8f6b-d9a1d04c57d8","added_by":"auto","created_at":"2022-03-01 21:43:57","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":146925,"visible":true,"origin":"","legend":"\u003cp\u003eFramework of the classification Stacked Ensemble of ResNet models\u003c/p\u003e","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/49ec57dc0419d3a7d70ca1a7.png"},{"id":18754493,"identity":"eeb5657b-0535-4145-858e-6cbb22f39e64","added_by":"auto","created_at":"2022-03-01 21:40:58","extension":"png","order_by":10,"title":"Figure 10","display":"","copyAsset":false,"role":"figure","size":186541,"visible":true,"origin":"","legend":"\u003cp\u003eThe proposed integrated CAD framework. (a) Original mammogram with ground truth of mass (red), (b) Detected ROI of mass (yellow) superimposed on the original mammogram, (c) Detected ROI mass obtained with ground truth (red), (d) Output segmented binary mask of ROI mass, and (e) Segmented ROI mass\u0026nbsp;with marked tissue\u0026nbsp;\u003c/p\u003e\u003cp\u003e\u003cbr\u003e\u003c/p\u003e","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/c8f7d8cefb95b4f6f83f4797.png"},{"id":18753516,"identity":"5215e563-9a35-4f49-95e4-5ba0603d10cf","added_by":"auto","created_at":"2022-03-01 21:34:58","extension":"png","order_by":11,"title":"Figure 11","display":"","copyAsset":false,"role":"figure","size":406751,"visible":true,"origin":"","legend":"\u003cp\u003eSamples of entire mammograms and ROI of mass detected and segmented from different mammography datasets with ground-truth of location and contour of mass in red\u003c/p\u003e","description":"","filename":"floatimage11.png","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/70b62ae2e9bc6a8d414de051.png"},{"id":18754643,"identity":"89d33044-64ee-4d94-9cf9-bf39f30a5769","added_by":"auto","created_at":"2022-03-01 21:44:01","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":875366,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/b7279c30-6023-4d49-86ff-66cf524c5660.pdf"},{"id":18754492,"identity":"e120ecfd-0273-4717-9406-f44fe21a9db2","added_by":"auto","created_at":"2022-03-01 21:40:58","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":573309,"visible":true,"origin":"","legend":"","description":"","filename":"SupplementaryInformation.pdf","url":"https://assets-eu.researchsquare.com/files/rs-1389924/v1/d25462c223c57d7c1765959b.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"An integrated Framework for Breast Mass Classification and Diagnosis using Stacked Ensemble of Residual Neural Networks","fulltext":[{"header":"Introduction","content":"\u003cp\u003eOver years, breast cancer has remained the most frequently diagnosed non-skin cancer and the leading cause of death among females with a rate of 32% of total cancer cases [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. According to the American Cancer Society, it is estimated that over 290,000 new cases will be reported and 43,780 women will die from breast cancer in 2022 [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Early detection and diagnosis of breast cancer is the most effective way to treat this disease and reduce the mortality rate [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eMammography has been proven the most reliable and preferred tool used by radiologists to screen and investigate suspicious breast lesions [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. However, with the increase in the number of daily-screened mammograms, an efficient diagnostic methodology is necessary to assist doctors in the timely procedure of breast cancer. Thus, computer-aided diagnosis (CAD) systems, that perform computational image analysis, could provide a second suggestion and reading to the final examination of the experts regarding the presence of breast cancer [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eA completely integrated CAD system would start its first stage, the detection and localization of suspicious lesions and distinguishing between their types, i.e. mass, calcification, architectural distortion, etc. Then, at a second stage, the CAD system should perform a segmentation of the obtained region of interest (ROI) surrounding the breast lesion in order to recognize its anatomical contour and remove its tissue background without losing its shape precision. Finally, diagnostic information can be extracted regarding the lesion\u0026rsquo;s pathology to classify the decided lesion as either malignant or benign, and identify its characteristics such as tumor grading using Breast Imaging Reporting and Data System (BI-RADS) score, and shape categorization. As the automated procedure relies on connected stages, each output information must be generated precisely to generate a fast and accurate final decision. Therefore, different algorithms have been widely implemented in CAD systems, and the most common used are conventional machine learning classifiers and threshold-based methods that are based on handcrafted features [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWith the practical challenges that breast tumors offer due to their variation in size, shape, location, and texture, there has been a significant need to improve the overall performance of CAD systems and reduce false positive and negative cases. With the recent progress in computers and their enhanced computational capacity and speed, deep learning methodology has been broadly suggested in biomedical applications and particularly in CAD systems for mammography [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn the last two decades, deep learning has shown a growing success in many computer vision tasks and has proven a capability to overcome complex problems in the medical imaging domain. As a result, several works have been suggested and applied particularly in mammography, such as for tumors detection [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e], breast lesions segmentation [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e], and classification [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eRecently, deep learning models have exceeded the simple adaptation of convolutional neural networks (CNN) algorithms to present several advanced architectures that outperformed the image classification results. In fact, the CNN architecture model was initially suggested for image classification and has been the base of many popular state-of-the-art architectures such as ResNet, AlexNet, EfficientNet, and VGG, etc. Consequently, many works have studied and applied the recent classification models for breast lesions classification, and have been employed in CAD systems in different methodologies such as using ensemble learning [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], transfer learning [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e], and fusion modeling [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e].\u003c/p\u003e \u003cdiv id=\"Sec2\" class=\"Section2\"\u003e \u003ch2\u003eTechnical Background\u003c/h2\u003e \u003cp\u003eSeveral research studies have attempted to suggest machine learning methods for computer-aided diagnosis (CAD) systems to assist experts in their final diagnostic decisions and have focused on improving results of breast mass classification in digital mammography. In this context, Dhahri et al. [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e] used a Tabu search to select the most significant features and then fed them into a K-Nearest Neighbors (KNN) algorithm to classify breast lesions into malignant or benign.\u003c/p\u003e \u003cp\u003eSince their development, many studies have given more attention to incorporating deep learning methods in CAD systems as they showed better efficiency than traditional CAD systems, which require extensive features extraction. For instance, an end-to-end approach was developed by Shen et al. [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e] to classify digital mammograms into cancer or normal. The work presented a modern CNN structure using VGG network and the residual network (ResNet), and achieved an area under the roc curve (AUC) of 0.91 on the CBIS-DDSM dataset and an AUC of 0.98 on the INbreast dataset. Another end-to-end model, called DiaGRAM, was built by Shams et al. [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e] that combined CNN and Generative Adversarial Networks (GAN). The work was conducted to classify mammograms as benign or cancerous and showed an accuracy of 89% on the DDSM dataset and 93.5% on the INbreast dataset. An improved deep learning method, called the DenseNet-II model, was invented in a work by Li et al. [\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e] for classification of benign and malignant mammograms. The model was applied on a private collection of mammograms and reached an accuracy of 94.55%. Accordingly, a model for mass classification was proposed by Zhang et al. [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] that fused texton features with deep CNN features and achieved an accuracy of 94.30% on the CBIS-DDSM dataset. In another work by Muramatsu et al. [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e], a CNN model\u0026rsquo;s performance was improved by adding synthetic data generated from lung nodules in computed tomography (CT) using cycle GAN. The classification performance was tested on a DDSM dataset and achieved an accuracy of 81.4%. Recently, Chakravarthy et al. [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e] proposed a customized method that integrated deep learning with an extreme learning machine (ELM) for classifying abnormal ROI images into malignant or benign. The proposed work achieved a maximum accuracy of 97.19% on DDSM, 98.13% on the Mammographic Image Analysis Society (MIAS) dataset and 98.26% on INbreast datasets. In a recent work by Khan et al. [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e], multi-view feature fusion (MVFF) based-CAD system was implemented to increase the performance of CNN by combining information of four views of mammograms in order to classify them into malignant or benign with an AUC of 0.84 on the CBIS-DDSM and mini-MIAS databases.\u003c/p\u003e \u003cp\u003eMoreover, Kumar et al. [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e] suggested a classification framework for breast density using an ensemble of 4-class neural network classifiers. The work showed a classification accuracy of 90.8% on the DDSM dataset. Besides ensemble learning methodology, transfer learning was also adapted with deep learning techniques to develop an approach for differentiation between benign and malignant breast cancer. Hence, in a work by Alkhaleefah et al. [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e], double-shot transfer learning (DSTL) was used by fine-tuning various pre-trained networks once on an ImageNet dataset, and another time on a larger dataset similar to the target dataset. The method was trained on the CBIS-DDSM and showed a better performance than single-shot transfer learning with an average AUC of 0.99 on the MIAS dataset and 0.94 on the BCDR dataset. Similarly, Falcon\u0026iacute; et al. [\u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e] used transfer learning on a NasNet Mobile model and fine tune on VGG models to classify mammogram images according to the BI-RADS scale that achieved an accuracy of 90.9% on the INbreast dataset. Recently, a work by Medeiro et al [\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] combined DenseNet201 and multi-perceptron layer (MLP) models to classify the pathology within BI-RADS levels 3 and 4 for malignancy of breast masses. The model achieved an accuracy of 63% surpassing the performance of a human expert by 9.0%.\u003c/p\u003e \u003cp\u003eTo accomplish an efficient mass classification and diagnosis procedure, researchers have shown that capturing texture and morphological characteristics could help doctors understand the nature of the breast tumor and assess its malignancy scale. For instance, a research by Bi et al. [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e] showed that the probability of malignancy is highly correlated with the shape and morphology of a breast lesion. Therefore, several works have incorporated the segmentation stage to provide a complete, significant diagnosis. In a previous work by Tsochatzidis et al. [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] modified convolutional layers of a CNN to integrate both input images and their corresponding segmentation maps in order to improve the diagnosis of breast cancer. The method was applied on DDSM-400 and CBIS-DDSM datasets and achieved a diagnosis performance of AUC of 0.89 and 0.86. Similarly, a dual convolutional neural network was suggested by Li et al. [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e], which computed the mass segmentation and simultaneously predicted the diagnosis results. The model contributed an improvement to the mass segmentation and cancer classification problem at the same time and achieved an AUC of 0.85 on the DDSM dataset and 0.93 and the INbreast dataset.\u003c/p\u003e \u003cp\u003eRecently, most of the developed CAD systems have automated the breast cancer diagnosis procedure that gets an entire mammogram image and returns the final diagnosis. Thus, many studies have integrated the first stage of identifying the suspicious region of breast lesions and based on its automated output, performed the segmentation and classification tasks. For instance, Sarkar et al. [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e] proposed an automated CAD system that detects suspicious regions of potential lesions using a deep hierarchical prediction network and then classifies them into mass or non-mass, and finally into malignant or benign using a CNN structure. The work was tested and achieved an accuracy of 98.05% on the DDSM dataset and 98.14% on the INbreast dataset. Another fully automated system by Dhungel et al. [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e] for breast mass classification integrated mass detection and segmentation in a complete CAD system. The methodology used a multi-scale deep belief network (m-DBN) classifier followed by a cascade of CNNs and random forest classifiers for false positive reduction for mass detection, a conditional random field (CRF) for mass segmentation, and a multi-view deep residual neural network (mResNet) for mass classification. The proposed work achieved an AUC of 0.8 on the INbreast dataset. Another recent work by Singh et al. [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e] presented automatic workflow that detects breast tumor regions from mammograms using the Single Shot Detector (SSD), and then outlines its segmented mask using conditional Generative Adversarial Network (cGAN) that was finally used for shape classification using a CNN. The framework achieved an overall accuracy of 80% for the shape classification. Similarly, Al-Antari et al. [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e] proposed a fully integrated CAD system for digital mammograms via deep learning techniques. It started with a mass detection using the You-Only Look Once (YOLO) architecture model, then performed a mass segmentation on the detected regions using Full resolution convolutional network (FrCN), and finally classified the detected and segmented masses into benign or malignant using a CNN model. The entire framework had an overall classification accuracy of 95.64% on the INbreast dataset. The mass classification step was differently solved in a recent work of Al-Antari et al. [\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e] that separately adopted three conventional deep learning models including regular feedforward CNN, ResNet-50, and InceptionResNet-V2. The work achieved a maximum accuracy of 95.32% on the INbreast dataset.\u003c/p\u003e \u003cp\u003eInspired by the continuous success of the CNN model and its variations for breast mass classification, we propose a stacked ensemble of residual network (ResNet) models to classify and diagnose previously detected and segmented mass lesions. The proposed model uses three different architectures of the ResNet model, ResNet50V2, ResNet101V2, and ResNet152V2 that are transferred and fine-tuned on our mammography datasets. The models\u0026rsquo; layers are stacked together and reconfigured into an entire model for an overall classification and diagnosis of 1) the pathology as malignant or benign; the BI-RADS category as assessment score from 2 to 6; and 3) the lesions\u0026rsquo; shape as round, oval, lobulated, or irregular.\u003c/p\u003e \u003cp\u003eThe presented work will serve as the last stage of an integrated framework for a breast cancer CAD system. The previous stages were proposed in recent works by Baccouche et al. where detection and classification step was first applied using a YOLO-based fusion model to localize and identify suspicious breast lesions as mass or classification [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e], and then using only the detected masses, a Connected-UNets model was suggested for breast mass segmentation and improved with combining real and synthetic data generated by CycleGAN model [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn this paper, we conduct the final stage in a CAD system, breast mass classification and diagnosis, using a stacked ensemble of neural networks models. The proposed methodology completes our recent works conveyed for breast lesions detection and classification from entire mammograms, and followed by a breast mass segmentation step. The work presents an integrated framework for CAD system for breast cancer as the performance relies on three connected stages, and the current step generates the final decision about the breast mass\u0026rsquo; pathology (i.e. benign or malignant), its BI-RADS category (i.e. score from 2 to 6), and its shape (i.e. oval, round, lobulated, or irregular). The paper is inspired by ensemble models learning and fusion modeling that showed high efficiency in many recent studies. The suggested methodology was performed on two most popular public mammography datasets: Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and on a private collection of mammograms.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eAll experiments for the proposed methodology were conducted using Python 3.6 on a PC with the following specifications: Intel(R) Core (TM) i7-8700K processor with 32 GB RAM, 3.70 GHz frequency, and one NVIDIA GeForce GTX 1090 Ti GPU.\u003c/p\u003e\n\u003cdiv\u003e\n \u003ch2\u003eData preparation\u003c/h2\u003e\n \u003cp\u003eMammograms are often collected using a scanning machine of digital X-ray mammography that usually compresses the breast and consequently it degrades the images. Therefore, we apply preprocessing techniques to remove the additional noise and correct the data using a histogram equalization that smooths the pixel distribution. Furthermore, the pre-trained ResNet models require input image size of 224 x 224; therefore, we resize the detected and segmented ROIs from 256 x 256 using an inter-area resampling interpolation. Finally, all images are normalized to a range of [0, 1]. Samples of input data for each classification class are illustrated in Figure.1 where ROIs are distributed according to different class labels from the mammography datasets.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eEvaluation metrics\u003c/h2\u003e\n \u003cp\u003eAll classification tasks are evaluated overall using the accuracy, and area under curve (AUC) that reflect the performance of the model while considering the unbalanced mammography datasets. Particularly, for pathology classification, which presents a binary-classes case, we use three additional metrics called sensitivity, specificity scores, and F1-score, as shown in Eq.\u0026nbsp;1, Eq.\u0026nbsp;2, and Eq.\u0026nbsp;3. The F1-score is a coefficient that represents a harmonic average between the specificity and sensitivity, where its maximum score of 1 indicates perfect specificity and sensitivity and of 0 the worst performance. Moreover, the accuracy score is a rate of correct predictions over all cases as detailed in Eq.\u0026nbsp;4 where TP, TN, FP, and FN are defined per predicted class to represent the number of true positive, true negative, false positive, and false negative predictions, respectively.\u003c/p\u003e\n \u003cp\u003e\u003cspan\u003e\\(Sensitivity=\\frac{TP}{TP+FN}\\)\u003c/span\u003e (1)\u003c/p\u003e\n \u003cp\u003e\u003cspan\u003e\\(Specifictity=\\frac{TN}{TN+FP}\\)\u003c/span\u003e (2)\u003c/p\u003e\n \u003cp\u003e\u003cspan\u003e\\(F1-score=\\frac{2\\times TP}{2\\times TP+FP+FN}\\)\u003c/span\u003e (3)\u003c/p\u003e\n \u003cp\u003e\u003cspan\u003e\\(Accuracy=\\frac{True prediction}{Total cases}=\\frac{TP+TN}{TP+FP+TN+FN}\\)\u003c/span\u003e (4)\u003c/p\u003e\n \u003cp\u003eIn pathology classification, positive refers to the malignant class and negative refers to the benign class. In BI-RADS category and shape classification, macro averaging is used to compute the accuracy and the AUC scores. Consequently, a confusion matrix can be driven from these measurements to show the tradeoff between the true and predicted class labels.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eHyperparameters tuning\u003c/h2\u003e\n \u003cp\u003eExtensive experiments with different variations in hyperparameters were conducted to select the best parameters for the base ResNetV2 model. Considering their effect on the classification performance, only hyperparameters detailed in Table.1 were tuned to select the best configured network that outperforms the evaluated networks on all mammography datasets.\u003c/p\u003e\n \u003cp\u003eTable.1: Hyperparameters for the ResNetV2 base model\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eHyperparameters\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eValues explored\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDescription\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBatch size\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e32, 64\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003emini-batch training size\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEpochs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e20, 30, 50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNumber of training epochs\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDropout\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0%, 20%, 30%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e% of neuron of hidden layers \u0026ldquo;dropped\u0026rdquo; for regularization\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLR\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e10\u003csup\u003e\u0026minus;\u0026thinsp;1\u003c/sup\u003e, 10\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e, 10\u003csup\u003e\u0026minus;\u0026thinsp;3\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eLearning rate for the Adam optimizer\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eSmoothing\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0% 20%, 25%\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e% of label smoothing for the loss function\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eFor all datasets, we randomly split images for each class into groups of 80% for training, 20% divided equally between testing and validation sets. In each experiment, the same trainable parameters were used and each hyperparameter was varied accordingly. For all datasets and classification tasks, we used Adam optimized and evaluation was reported with a weighted accuracy score to reflect the class imbalance during the training and testing. The loss function was employed according to the classification task, a Binary Cross-entropy function for binary classes and Categorical Cross-entropy for multiple classes. In both cases, a label smoothing technique for regularization to help overcome overfitting and provide a generalized model. The technique works by explicitly updating the labels during the loss function and decreasing the model\u0026rsquo;s confidence when it starts diverging [45]. In addition, training was monitored using a method that reduces the learning rate if the accuracy stops improving. Thus, we applied the stated strategy with a factor of 0.5 when the accuracy did not improve after two iterations. Conclusively, the best evaluation was reported with a batch size of 32, 30 epochs, a dropout rate of 30%, a learning rate of 10\u003csup\u003e\u0026minus;\u0026thinsp;2\u003c/sup\u003e, and a smoothing label of 25%.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eQuantitative classification results\u003c/h2\u003e\n \u003cp\u003eThe proposed breast mass classification model was trained and compared to single base models for each presented task on the different mammography datasets. We also compared the stacked ensemble of models to a conventional average of different models\u0026rsquo; weights with an XGBoost classifier.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003ePathology classification\u003c/h2\u003e\n \u003cp\u003eAs shown in Table.2, Table.3, and Table.4, the pathology classification results are compared between different models respectively for CBIS-DDSM, INbreast, and private datasets. It is reasonable to mention that because the private dataset includes only malignant cases, we trained and tested the model on a combination of all datasets.\u003c/p\u003e\n \u003cp\u003eTable.2: Pathology classification results on the CBIS-DDSM dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"6\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSensitivity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSpecificity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1-score\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e89.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.9\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e93.57\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e92.11\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e91.04\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.85\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.98\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e95.13\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.93\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.97\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.95\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.95\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eTable.3: Pathology classification results on the INbreast dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"6\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSensitivity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSpecificity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1-score\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e98.52\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e95.58\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.9\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e96.6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e97.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.96\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.98\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e99.2\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.0\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.98\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.99\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.99\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eTable.4: Pathology classification results on the Private dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"6\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSensitivity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eSpecificity\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eF1-score\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e92.83\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e94.18\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e94.60\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e94.89\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e95.88\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.93\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.97\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.95\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.96\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eThe comparative results show that the proposed stacked ensemble of models performs better than the base ResNet models having different numbers of deep layers (i.e. ResNet50V2, ResNet101V2 and ResNet152V2). Accordingly, our proposed methodology outperformed the average ensemble of models with an XGBoost classifier that performed slightly better than individual models. We clearly notice a high accuracy of 95.13% on the CBIS-DDSM dataset, 99.2% on the INbreast dataset, and 95.88% on the private dataset. Besides, our proposed model achieved a high sensitivity rate of 0.93 on the CBIS-DDSM dataset, 1.0 on the INbreast dataset, and 0.93 on the private dataset. Consequently, the results emphasize generally the advantage of ensemble learning technique on improving the classification performance, and particularly the improvement achieved by the stacking method using deep learning models. Moreover, the pathology classification performance was compared against the different models using the AUC over the test sets of all datasets. Figure.2 shows plots of the Receiver Operating Characteristic (ROC) curves of the True positive Rate (TPR) against the False Positive Rate (FPR), and we clearly notice that the proposed model outperformed all experimental techniques with an AUC of 0.95 for the CBIS-DDSM dataset, 0.99 for the INbreast dataset, and 0.96 for the private dataset.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eBI-RADS category classification\u003c/h2\u003e\n \u003cp\u003eResults shown in Table.5, Table.6 and Table.7 for BI-RADS category classification illustrate the comparison between different models for all mammography datasets. As mentioned in the section on Datasets description, each dataset has different class labels that vary from category 2 to category 6.\u003c/p\u003e\n \u003cp\u003eTable.5: BI-RADS category classification results on the CBIS-DDSM dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e80.07\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e77.12\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e80.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e79.48\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.85\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e83.84\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.94\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eTable.6: BI-RADS category classification results on the INbreast dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e98.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e98.0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e96.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e97.1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.99\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e99.0\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e1.00\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eTable.7: BI-RADS category classification results on the Private dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e91.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e92.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e94.25\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e92.95\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e96.08\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.95\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eThe classification results presented above demonstrate a clear improvement of the performance using our proposed stacked ensemble of models comparing to the basic models with an accuracy of at least 3.78% on the CBIS-DDSM dataset, 1% on the INbreast dataset, and 1.83% on the private dataset. Moreover, our methodology achieved a better AUC score than the average ensemble model with an XGBoost classifier where we notice a high AUC of 0.94 for the CBIS-DDSM dataset, 1.00 on the INbreast dataset, and 0.95% on the private dataset. This can be confirmed with a visual comparison of ROC curve plots between employed models as illustrated in Figure.3.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eShape classification\u003c/h2\u003e\n \u003cp\u003eLastly, the proposed model was trained on the CBIS-DDSM dataset for classifying the shape of breast masses, as it is the only dataset that possesses shape annotation by experts. Equivalently, all trained models were tested, and comparison is shown in Table.8. Undoubtedly, our suggested stacked ensemble of models had the highest accuracy score of 90.02% among the employed models, which improved the performance of separate models notably with 1.7% and remarkably with 10.66% compared to the average ensemble of models with an XGBoost classifier.\u003c/p\u003e\n \u003cp\u003eTable.8: Shape classification results on the CBIS-DDSM dataset\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eModel\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel1: ResNet50V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e75.51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.90\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel2: ResNet101V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e89.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eModel3: ResNet152V2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e88.32\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAverage Weights of Model1, Model2 and Model3\u0026thinsp;+\u0026thinsp;XGBoost Classifier\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e79.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0.84\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eStacked Ensemble of models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e90.02\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e\u003cstrong\u003e0.98\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003eFurthermore, Figure.4 presents a comparison of ROC curve plots for the different employed models and represents the AUC score accordingly. We clearly notice that our proposed model had the highest AUC of 0.98 among the presented models, which was close to the ResNet152V2 performance but with a slightly better accuracy rate.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eQualitative classification results\u003c/h2\u003e\n \u003cp\u003ePrevious comparison results highlighted that our proposed methodology yielded the best results for the different classification tasks. Consequently, we analyzed the classification prediction between different datasets using the confusion matrix that summarizes the results across the class labels.\u003c/p\u003e\n \u003cp\u003eFigure.5, Figure.6 and Figure.7 respectively present the normalized confusion matrix plots for each classification problem. As indicated below, the INbreast dataset had the best pathology classification tradeoff between malignant and benign classes, and this can be explained with the high-quality resolution of the mammograms collected in FFDM format that helps distinguish between the two class labels. The private dataset had also a remarkable confusion matrix with close recall and precision scores, which were similar to the CBIS-DDSM dataset\u0026rsquo;s performance.\u003c/p\u003e\n \u003cp\u003eMoreover, the INbreast dataset had the best BI-RADS categorization tradeoff with a notable prediction per class from 0.92 to 1.0. Concerning the private dataset, it has only two BI-RADS categories 4 and 5, and we notice a similar satisfying confusion matrix with prediction scores of 0.93 and 0.96. The CBIS-DDSM dataset has a slightly worse tradeoff for the BI-RADS category classification and this is due to the low resolution of deteriorated ROI images from the digitized X-rays mammograms. In fact, the confusion matrix shows values from 0.80 to 0.89 and we clearly notice that the four categories have a close prediction score due to the similarity of the pixel distribution caused by the quality presented in the public dataset.\u003c/p\u003e\n \u003cp\u003eFinally, the confusion matrix for the shape classification showed an overall sufficient tradeoff between the class labels. We clearly observe similar predicted results for the irregular and lobulated cases with a maximum value of 0.93, and this can be interpreted by the close appearance of the two lesions\u0026rsquo; shape. However, oval and round cases had worse results, and in particular, the round class label had a performance score of 0.73.\u003c/p\u003e\n \u003cp\u003eAdditionally, we reported the classification results of the final integrated CAD system using the segmentation step and compared it to results of the proposed method without using the segmented ROIs. Table.9 shows a better classification performance for each task on all mammography datasets, where the pathology classification demonstrated improvement of 4.26% on the CBIS-DDSM dataset, 4% on the INbreast dataset, and 5.5% on the private dataset. Accordingly, the BI-RADS category classification presented enhanced performance using the detected and segmented images with accuracy difference of 3.69% on the CBIS-DDSM dataset, 1.5% on the INbreast dataset, and 0.38% on the private dataset. Finally, the CBIS-DDSM dataset had improved the shape classification results with 4.36% accuracy. Consequently, it is clearly observed that the CAD system with the integrated detection and segmentation stages achieved much better results for the classification and diagnosis of breast masses.\u003c/p\u003e\n \u003cp\u003eTable.9: Comparison of classification performance (accuracy %) using the proposed CAD system with and without mass segmentation\u0026nbsp;\u003c/p\u003e\n \u003ctable border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eBI-RADS Category\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003eShape\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system without mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system with mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system without mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system with mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system without mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eCAD system with mass segmentation\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e90.87\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e95.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e81.69\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e85.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e85.66\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e90.02\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e95.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e99.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e97.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e90.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e95.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e95.70\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e96.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\u003cbr\u003eFinally, a comparison of results of the latest state-of-the-art methods and similar models to classify the breast masses is listed in Table.10. Our proposed CAD system that integrates the previous detection and segmentation steps and the current proposed classification framework outperformed the previous deep learning models applied for pathology, BI-RADS category, and shape classification.\u003ctable border=\"1\"\u003e\u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003eCompared with other techniques that used segmented ROIs, we exceeded the performance of the work by Falcon\u0026iacute; et al. [20] that only achieved an accuracy of 78.4% using the MobileNet model on the CBIS-DDSM dataset. On the other hand, we also outperformed the work of Alkhaleefah et al. [33] even though they did not use segmented input images and reported an accuracy of 93.47%. Moreover, recent works on the INbreast dataset were all surpassed where the highest accuracy of 98.26% was reported by Chakravarthy et al. [30] using the ICS-ELM algorithm on original ROI masses. We also reported better accuracy score for the pathology classification applied on segmented ROIs from the INbreast dataset, where Al-Antari et al. [42] only achieved an accuracy of 95.64% and an AUC score of 0.94. The results of the BI-RADS categorization also outperformed the previous works on CBIS-DDSM dataset with the work suggested by Medeiros et al. [35] that applied DenseNet201 on original ROI masses and only achieved an accuracy of 63.4%. No previous paper applied the BI-RADS category classification on segmented images and therefore we could not compare with our proposed work. Accordingly, our method surpassed the performance on the INbreast dataset by the work of Falcon\u0026iacute; et al. [34] that only reported an accuracy of 90.9% using NasNet and VGG models. Lastly, our methodology gained the best shape classification performance compared to a recent work of Singh et al. [41] that applied a CNN model on a similar dataset, and it is reasonable to say that this reviewed work is the only comparable work that applied shape classification on detected and segmented ROIs but only achieved an accuracy of 80%.\u003c/p\u003e\n \u003cp\u003eTable.10: Comparison of the proposed methodology and state-of-the-art methods\u003c/p\u003e\n \u003cdiv\u003e\u0026nbsp;\u003ctable border=\"1\"\u003e\n \u003ccolgroup cols=\"8\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eReference\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eYear\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eMethod\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eClassification\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eImages Segmented\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAccuracy\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAUC\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[40]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2017\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003emulti-view deep residual neural network (mResNet)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e--\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.80\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[42]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2018\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eEnsemble of AlexNet-based CNN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.64\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[20]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2019\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eMobileNet and NasNet\u0026thinsp;+\u0026thinsp;fine tuning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e78.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e--\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[21]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eVGG16\u0026thinsp;+\u0026thinsp;fine tuning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e84.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.84\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[30]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eImproved Crow-Search Optimized Extreme Learning Machine (ICS-ELM) algorithm\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e98.26\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e--\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[33]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eAlexNet, VGG,\u003c/p\u003e\n \u003cp\u003eGoogLeNet, ResNet\u003c/p\u003e\n \u003cp\u003e+ fine tuning\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e93.47\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[35]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDenseNet201\u0026thinsp;+\u0026thinsp;MLP\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBI-RADS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e63.4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e--\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[34]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNasNet\u0026thinsp;+\u0026thinsp;fine tuning on VGG16 and VGG19\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eBI-RADS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e90.9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.99\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[41]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2020\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCNN\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eShape\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e80\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.80\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e[38]\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2021\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eDualCoreNet: Texture and shape features fusion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e--\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.93\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"7\"\u003e\n \u003cp\u003eProposed CAD\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\" rowspan=\"7\"\u003e\n \u003cp\u003e2022\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" rowspan=\"7\"\u003e\n \u003cp\u003eStacked Ensemble of ResNet models\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.13\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99.20\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.99\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e95.88\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003eBI-RADS\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\" rowspan=\"3\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e85.38\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.94\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e99\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e1.00\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e96.08\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.95\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eShape\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eYes\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e90.02\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e0.98\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eDeep learning models have recently revealed remarkable success in breast mass classification and diagnosis for many CAD systems. The CNN architecture model was mostly modified on many proposed studies and combined with other recent techniques such as transfer learning and ensemble models learning for a better classification performance.\u003c/p\u003e \u003cp\u003eIn this study, we have implemented a stacked ensemble of ResNet models to classify breast masses as malignant or benign and diagnose their BI-RADS category assessment with a score from 2 to 6 and their shape as oval, round, lobulated or irregular. The results of the proposed methodology showed the classification performance\u0026rsquo;s improvement compared to the individual architectures and to the other methods applied on the existing benchmark datasets. Table.10 shows that we achieved the highest pathology classification performance on the two public datasets: CBIS-DDSM with an accuracy of 95.13% and an AUC score of 0.95, and INbreast with an accuracy of 99.20 and an AUC score of 0.99. Furthermore, we surpassed the results of other models for the BI-RADS categorization on the CBIS-DDSM dataset with an accuracy of 85.38% and an AUC score of 0.94, and on the INbreast dataset with an accuracy of 99% and an AUC score of 1.0. We also reported the highest results on the shape classification for the CBIS-DDSM dataset with an accuracy of 90.02% and an AUC score of 0.98.\u003c/p\u003e \u003cp\u003eComparing with the similar frameworks that applied the presented classification tasks on segmented ROI masses, our model outperformed the MobileNet and NasNet models [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] for the pathology classification on the CBIS-DDSM dataset and the Ensemble of AlexNet-based CNN model [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e] on the INbreast dataset. Moreover, the shape classification achieved better results on a similar dataset DDSM that was evaluated with an individual CNN model [\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e]. As a result, the stacking model technique provided an efficient way to learn from various depths of neural networks and combine them in another neural network classifier model to benefit from the different weights that were trained individually.\u003c/p\u003e \u003cp\u003eThe work integrated our recent works of the YOLO-based fusion models [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] and the Connected-UNets model [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e] that generated the detected and segmented ROIs of breast masses. Indeed, an increase of performance using the segmented ROIs, as shown in Table.9, indicates the advantage of masking the background tissues from the tumors\u0026rsquo; boundaries to help improve the overall classification and diagnosis, and decrease the false positive and negative rates. Limitations of the proposed methodology can occur on the long training time of 0.74s per epoch, which is due to the high number of trainable parameters and computations of the ResNetV2 model.\u003c/p\u003e \u003cp\u003eIn conclusion, this work presents the final stage of an integrated framework for a breast cancer CAD system via deep learning models. The three stages of detection, segmentation and classification aim to provide a complete clinical tool that can assist radiologists with a second suggestion with an automated mass tumor diagnosis. Future works can include combining different mammography datasets and improving the long training of deep learning models for the classification task.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eIn this study, we propose a stacked ensemble of models to classify and diagnose detected and segmented breast masses. The base model comes from the ResNet architecture and its variations. Our methodology employs different strategies: transfer learning, stacked ensemble learning, and image data augmentation.\u003c/p\u003e\n\u003cdiv\u003e\n \u003ch2\u003eResNet base model: transfer learning and fine-tuning\u003c/h2\u003e\n \u003cp\u003eSince its introduction, ResNet is a deep CNN architecture suggested by He et al. [46] that has been one of the recent architectures that has known common success in medical imaging applications [47, 48]. ResNet uses residual blocks with skip connection between layers to bypass a few convolution layers at a time. This architecture accelerated the convergence of a larger number of deep layers, and consequently it has been found efficient to provide a compact representation of input images and improve the classification task performance [22]. The ResNet has some common architectures such as ResNet-50, 101, and 152, [37] which indicate the number of deep layers. Alternatively, ResNet architecture presented an improved version ResNetV2 by He et al. [49], where the last ReLU was removed to clear the shortcut path using a simple identity connection as shown in Supplementary Figure.1.\u003c/p\u003e\n \u003cp\u003eOur methodology employs three pre-trained ResNetV2 architectures, detailed below in Supplementary Table.1. Training a deep learning model often requires a large amount of annotated data that helps optimizing the high number of parameters and computations needed in the architecture. However, a limited size of medical imaging datasets are usually available that suffer from either missing labels or imbalanced data distribution. To overcome these challenges, transfer learning has been a common solution used in many recent medical image application [50, 51] by training a model on a large and diverse dataset (i.e. ImageNet, MSCOCO, etc.) to capture universal features like curves, edges, and boundaries in its early layers that are relevant for image classification. After that, the pre-trained model should be alerted and fine-tuned on a custom and specific dataset to reflect the final classification. This procedure provides a fast and generalizable training of small datasets and avoids the overfitting problem that deep learning commonly suffers from.\u003c/p\u003e\n \u003cp\u003eAs Figure.8 indicates, we apply transfer learning to the base architecture ResNetV2 for our proposed methodology to become a TF-ResNetV2. In fact, the model was initially pre-trained on ImageNet, and then the first four residual blocks of layers were frozen with exception to the BN layers that needed to be retrained in order to improve the training convergence. After that, the entire architecture was modified by adding another FC layer with size of 1024, followed by a dropout regularization layer to maintain a generalization aspect for the training. A new final FC layer was placed according to the number of classes for each classification task and the entire TF-ResNetV2 is re-trained.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eStacked Ensemble of ResNet models for Breast mass classification\u003c/h2\u003e\n \u003cp\u003eEnsemble learning has been considered efficient to improve the classification task results. Combining weaker classifiers to create a better final classification prediction has been adopted by either bagging, boosting, or stacking models. While bagging is achieved by learning independently from different models and then averaging the predictions, boosting happens by sequentially learning from homogenous learners and iteratively combining them into a final model. On the other hand, stacking has been considered a way to learn different weak learners in parallel and combine them into a meta-model that is later trained to achieve the classification prediction [52].\u003c/p\u003e\n \u003cp\u003eWe propose a stacked ensemble of three different ResNet models to conduct our classification tasks. After removing the last FC layer of each ResNetV2 architecture, a two-layers network is considered as a meta-classifier model that concatenates the three models\u0026rsquo; layers, and stacks three different FC layers of size 1000, 100 and 10, coupled with activation functions Sigmoid and ReLU. As shown in Figure.9, after training independently ResNet50V2, ResNet101V2 and ResNet152V2, pre-trained weights of each model were extracted as images features of size 1024 based on previous layer predictions and considered as new input of the entire stacked ensemble of ResNet models for the final class prediction.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eIntegrated framework: Mass detection, segmentation and classification\u003c/h2\u003e\n \u003cp\u003eOur final framework should now be complete with all automated steps for breast cancer analysis and diagnosis. Therefore, as shown in Figure.10, the integrated framework detects and localizes breast masses in a first step using YOLO-based fusion models [23], which only requires an entire mammogram image and outputs bounding boxes around specious lesions. The model was evaluated and provided a maximum detection accuracy of 98.1% for mass lesions. The next step should segment the detected ROI of breast masses and generate a binary mask image where only the boundary of the lesions is visible. The second step is achieved using the proposed Connected-UNets model [44] that was improved by synthetic data, which is generated by CycleGAN. The segmentation step was conducted on ROI images scaled to an optimal size of 256 x 256 pixels.\u003c/p\u003e\n \u003cp\u003eEvaluation showed a high Dice score of 95.88% and Intersection over Union (IoU) of 92.27%. After that, the segmented and detected ROI of breast masses generated with masked tissue are used for the third and final classification step. The stacked ensemble of ResNet models is trained independently on the input ROI masses for each classification task to finally predict the pathology as either malignant or benign, the BI-RADS category with an assessment score between 2 to 6, and the shape as either round, oval, lobulated or irregular.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv\u003e\n \u003ch2\u003eDatasets\u003c/h2\u003e\n \u003cp\u003eSimilar to our previous works, we evaluate the proposed classification methodology on two public datasets, the CBIS-DDSM and INbreast datasets, and on an independent private dataset. The CBIS-DDSM dataset [53] is an updated and standardized version of the Digital Database for Screening Mammography (DDSM), where images were reviewed by radiologists to eliminate inaccurate cases and converted from Lossless Joint Photographic Experts Group (LJPEG). It contains 2907 mammograms from 1555 unique patients, where 1467 are mammograms with mass lesions acquired with two different views (i.e. MLO and CC). Original mammograms have an average size of 3000 x 4800 pixels and are associated with their pixel-level annotation and class labels (i.e. Pathology, BI-RADS category and Shape).\u003c/p\u003e\n \u003cp\u003eThe INbreast dataset [54] is a public database of Full-Field Digital Mammography (FFDM) images in DICOM format. It contains 410 mammograms from 115 unique patients where only 107 cases present mass lesions in both MLO and CC views. Original mammograms have an average size of 3328 x 4084 pixels and include pixel-level annotation and class labels (i.e. Pathology and BI-RADS category).\u003c/p\u003e\n \u003cp\u003eThe private dataset is a collection of mammograms from the National Institute of Cancerology (INCAN) in Mexico City and contains stage 3 and 4 of breast cancer with 389 cases from 208 unique patients having mass lesions. Images have an average of 300 x 700 pixels acquired from different views (i.e. CC, MLO, ML and AT), and include associated pixel-level annotation and class labels (i.e. Pathology and BI-RADS category). Figure.11 illustrates samples of original mammograms and their ROI masses compared to the detected and segmented ROI masses from different datasets.\u003c/p\u003e\n \u003cp\u003eAs the datasets were explored continuously during the previous studies, the original mammograms that included mass and calcification cases were used during the first step of detection and localization; therefore detected ROIs of only mass cases were retained for the second step of segmentation. It is fair to mention that some mammograms have multiple ROIs and hence the number of detected and segmented ROI masses used for the third step of classification and diagnosis may vary. Due to the limited amount of ROI masses in each dataset, raw ROIs data was augmented four times by rotating them with the angles \u0026Delta;\u0026theta; = {0\u0026deg;, 90\u0026deg;, 180\u0026deg;, 270\u0026deg;}, and transformed twice differently using the Contrast Limited Adaptive Histogram Equalization (CLAHE) method. Table.11 details the data distribution of each mammography dataset regardless of the class labels.\u003c/p\u003e\n \u003cp\u003eTable.11: General Datasets distribution\u0026nbsp;\u003c/p\u003e\n \u003ctable border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRaw MGs Data\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eRaw ROIs Data\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eAugmented Data (ROIs*6)\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1,467\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1,467\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e8,802\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e107\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e112\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e672\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e389\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e638\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3,828\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003eMoreover, each mammography dataset has a different quality of images in terms of pixel quality, existing annotated labels and class distribution, as detailed in Table.12, 13 and 14. In fact, only the CBIS-DDSM dataset includes true class labels for lesions\u0026rsquo; shape. Accordingly, the INbreast dataset indicates cases with BI-RADS score from 2 to 6, however the CBIS-DDSM dataset presents cases in BI-RADS category 2 to 5, and the private dataset has only malignant cases as it acquired breast cancer cases from only stage 3 and 4. Consequently, all mammograms from the private dataset fall into BI-RADS categories 4 and 5.\u003c/p\u003e\n \u003cp\u003eTable.12: Pathology class labels distribution\u0026nbsp;\u003c/p\u003e\n \u003ctable border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"2\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eBenign\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eMalignant\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM*\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4,500\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e4,302\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast**\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e150\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e522\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3830\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003ctfoot\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\"\u003e* Cases with Benign_without_callback are considered Benign\u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"3\"\u003e** Cases with BI-RADS score\u0026thinsp;\u0026gt;\u0026thinsp;3, are considered Malignant otherwise Benign\u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tfoot\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003eTable.13: Shape class labels distribution\u0026nbsp;\u003c/p\u003e\n \u003ctable border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"5\"\u003e\n \u003cp\u003eShape\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eRound\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eOval\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eLobulated\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eIrregular\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e804\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2,040\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e2,112\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003e3,846\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eNA\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003cp\u003e\u003c/p\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n \u003cp\u003eTable.14: BI-RADS category class labels distribution\u003c/p\u003e\n \u003cdiv\u003e\n \u003ctable border=\"1\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\" rowspan=\"2\"\u003e\n \u003cp\u003eDataset\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\" colspan=\"5\"\u003e\n \u003cp\u003eBI-RADS\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory 2\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory 3\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory 4\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory 5\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003e\u003cstrong\u003eCategory 6\u003c/strong\u003e\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eCBIS-DDSM\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e792\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1,938\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2,328\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e3,402\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eINbreast\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e144\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e126\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e276\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e48\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003ePrivate\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e1,627\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e2,195\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e0\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eDATA AVAILABILITY\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe public mammography dataset CBIS-DDSM generated and analyzed during the current study is available in the Cancer Imaging Archive, https://wiki.cancerimagingarchive.net/display/Public/CBIS-DDSM. The public mammography dataset INbreast generated and analyzed during the current study is available from the corresponding author In\u0026ecirc;s Domingues, Porto, Portugal, on reasonable request after signing a transfer agreement. The private mammography dataset generated during and analyzed during the current study is available from the corresponding author Cristian Castillo Olea through the oncologist Dr. Eric Ortiz in the National Institute of Cancerology, Mexico.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCODE AVAILABILITY\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe code for custom algorithms and data preprocessing is provided as part of the replication package. It was written in Python v3.6. It is publicly available as a git repository on GitHub at https://github.com/AsmaBaccouche/Stacked-Ensemble-of-Residual-Neural-Networks\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eACKNOWLEDGEMENTS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors would especially like to express their gratitude to the National Institute of Cancerology (INCAN) in Mexico City and particularly to Christian Castillo for providing the private mammography dataset. Thanks also to the radiologists Dr. Kictzia Yigal Larios and Dr. Raquel Balb\u0026aacute;s at FUCAM A. C., and Dr. Guillermo Peralta and Dr. N\u0026eacute;stor Pi\u0026ntilde;a at Cancer Center Tec100 by MRC International.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAUTHOR CONTRIBUTIONS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA.B conceived the idea, developed and implemented the methods. B.G-Z helped with formulating and validating the experiments and analysis. B.G-Z and A.S.E supervised the project. A.B wrote the paper. All authors reviewed and edited the manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCOMPETING INTERESTS\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eADDITIONAL INFORMATION\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eSupplementary information \u003c/strong\u003eThe online version contains supplementary material available Supplementary Information.pdf.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eFerlay, J. et al. Cancer statistics for the year 2020: An overview. \u003cem\u003eInternational Journal of Cancer\u003c/em\u003e\u003cem\u003e \u003c/em\u003e\u003cstrong\u003e149\u003c/strong\u003e\u003cem\u003e, 778-789 \u003c/em\u003e(2021).\u003c/li\u003e\n\u003cli\u003eAmerican Cancer Society. Cancer Statistics Center. http://cancerstatisticscenter.cancer.org. Accessed January 17, 2022.\u003c/li\u003e\n\u003cli\u003eDuffy, S. W. et al. Mammography screening reduces rates of advanced and fatal breast cancers: Results in 549,091 women. \u003cem\u003eCancer\u003c/em\u003e \u003cstrong\u003e126\u003c/strong\u003e, 2971-2979 (2020).\u003c/li\u003e\n\u003cli\u003eDibden, A., Offman, J., Duffy, S. W., \u0026amp; Gabe, R. Worldwide review and meta-analysis of cohort studies measuring the effect of mammography screening programmes on incidence-based breast cancer mortality. \u003cem\u003eCancers\u003c/em\u003e \u003cstrong\u003e12\u003c/strong\u003e, 976 (2020).\u003c/li\u003e\n\u003cli\u003eRahman, M. M. el al. Machine learning based computer aided diagnosis of breast cancer utilizing anthropometric and clinical features. \u003cem\u003eIrbm\u003c/em\u003e \u003cstrong\u003e42\u003c/strong\u003e, 215-226 (2021).\u003c/li\u003e\n\u003cli\u003eRamadan, S. Z. Methods used in computer-aided diagnosis for breast cancer detection using mammograms: a review. \u003cem\u003eJournal of healthcare engineering\u003c/em\u003e (2020).\u003c/li\u003e\n\u003cli\u003eYassin, N. I., Omran, S., El Houby, E. M., \u0026amp; Allam, H. Machine learning techniques for breast cancer computer aided diagnosis using different image modalities: A systematic review. \u003cem\u003eComputer methods and programs in biomedicine\u003c/em\u003e \u003cstrong\u003e156\u003c/strong\u003e, 25-45 (2018).\u003c/li\u003e\n\u003cli\u003eParamkusham, S., Thotempuddi, J., \u0026amp; Rayudu, M. S. Breast masses classification using contour shape descriptors based on Beam Angle Statistics. In \u003cem\u003e2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA)\u003c/em\u003e, 1340-1345. (IEEE, 2021).\u003c/li\u003e\n\u003cli\u003eLi, H., Meng, X., Wang, T., Tang, Y., \u0026amp; Yin, Y. Breast masses in mammography classification with local contour features. \u003cem\u003eBiomedical engineering online\u003c/em\u003e \u003cstrong\u003e16\u003c/strong\u003e, 1-12 (2017).\u003c/li\u003e\n\u003cli\u003eAl-Masni, M. A. et al. Simultaneous detection and classification of breast masses in digital mammograms via a deep learning YOLO-based CAD system. \u003cem\u003eComputer methods and programs in biomedicine \u003c/em\u003e\u003cstrong\u003e157\u003c/strong\u003e, 85\u0026ndash;94 (2018).\u003c/li\u003e\n\u003cli\u003eEltrass, A. S., \u0026amp; Salama, M. S. Fully automated scheme for computer‐aided detection and breast cancer diagnosis using digitised mammograms. \u003cem\u003eIET Image Processing\u003c/em\u003e \u003cstrong\u003e14\u003c/strong\u003e, 495-505 (2020).\u003c/li\u003e\n\u003cli\u003eSiddiqui, S. Y. et al. Intelligent breast cancer prediction empowered with fusion and deep learning. Computer, Materials and Continua \u003cstrong\u003e67\u003c/strong\u003e, 1033\u0026ndash;1049 (2021).\u003c/li\u003e\n\u003cli\u003eAly, G. H., Marey, M., El-Sayed, S. A., \u0026amp; Tolba, M. F. YOLO Based Breast Masses Detection and Classification in Full-Field Digital Mammograms. \u003cem\u003eComputer Methods and Programs in Biomedicine\u003c/em\u003e \u003cstrong\u003e200\u003c/strong\u003e, 105823 (2021).\u003c/li\u003e\n\u003cli\u003eAl-Antari, M. A., Han, S. M., \u0026amp; Kim, T. S. Evaluation of deep learning detection and classification towards computer-aided diagnosis of breast lesions in digital X-ray mammograms. \u003cem\u003eComputer methods and programs in biomedicine\u003c/em\u003e \u003cstrong\u003e196\u003c/strong\u003e, 105584 (2020).\u003c/li\u003e\n\u003cli\u003eAbdelhafiz, D., Bi, J., Ammar, R., Yang, C. \u0026amp; Nabavi, S. Convolutional neural network for automated mass segmentation in mammography. \u003cem\u003eBMC Bioinformatics\u003c/em\u003e \u003cstrong\u003e21\u003c/strong\u003e, 1\u0026ndash;19 (2020).\u003c/li\u003e\n\u003cli\u003eTaghanaki, S. A. et al. Deep semantic segmentation of natural and medical images: a review. \u003cem\u003eArtif. Intell. Rev.\u003c/em\u003e \u003cstrong\u003e54\u003c/strong\u003e, 137\u0026ndash;178 (2021).\u003c/li\u003e\n\u003cli\u003eJiao, Z., Gao, X., Wang, Y., \u0026amp; Li, J. A parasitic metric learning net for breast mass classification based on mammography. \u003cem\u003ePattern Recognition\u003c/em\u003e \u003cstrong\u003e75\u003c/strong\u003e, 292-301 (2018).\u003c/li\u003e\n\u003cli\u003eRasti, R., Teshnehlab, M., \u0026amp; Phung, S. L. Breast cancer diagnosis in DCE-MRI using mixture ensemble of convolutional neural networks. \u003cem\u003ePattern Recognition\u003c/em\u003e \u003cstrong\u003e72\u003c/strong\u003e, 381-390 (2017).\u003c/li\u003e\n\u003cli\u003eRampun, A., Scotney, B. W., Morrow, P. J., \u0026amp; Wang, H. Breast mass classification in mammograms using ensemble convolutional neural networks. In \u003cem\u003e2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom)\u003c/em\u003e, 1-6 (IEEE 2018).\u003c/li\u003e\n\u003cli\u003eFalcon\u0026iacute;, L. G., P\u0026eacute;rez, M., \u0026amp; Aguilar, W. G. Transfer learning in breast mammogram abnormalities classification with mobilenet and nasnet. In \u003cem\u003e2019 International Conference on Systems, Signals and Image Processing (IWSSIP)\u003c/em\u003e, 109-114 (IEEE, 2019).\u003c/li\u003e\n\u003cli\u003eFalconi, L. G., Perez, M., Aguilar, W. G., \u0026amp; Conci, A. Transfer learning and fine tuning in breast mammogram abnormalities classification on CBIS-DDSM database. \u003cem\u003eAdvances in Science, Technology and Engineering Systems\u003c/em\u003e \u003cstrong\u003e5\u003c/strong\u003e, 154-165 (2020).\u003c/li\u003e\n\u003cli\u003eRagab, D. A., Attallah, O., Sharkas, M., Ren, J., \u0026amp; Marshall, S. A framework for breast cancer classification using multi-DCNNs. \u003cem\u003eComputers in Biology and Medicine\u003c/em\u003e \u003cstrong\u003e131\u003c/strong\u003e, 104245 (2021).\u003c/li\u003e\n\u003cli\u003eBaccouche, A., Garcia-Zapirain, B., Castillo Olea, C. \u0026amp; Elmaghraby, A. S. Breast lesions detection and classification via yolo-based fusion models. \u003cem\u003eComputer. Material. Contina \u003c/em\u003e\u003cstrong\u003e69\u003c/strong\u003e, 1407\u0026ndash;1425 (2021).\u003c/li\u003e\n\u003cli\u003eDhahri, H., Rahmany, I., Mahmood, A., Al Maghayreh, E., \u0026amp; Elkilani, W. Tabu search and machine-learning classification of benign and malignant proliferative breast lesions. \u003cem\u003eBioMed research international\u003c/em\u003e (2020).\u003c/li\u003e\n\u003cli\u003eShen, L. et al. Deep learning to improve breast cancer detection on screening mammography. \u003cem\u003eScientific reports\u003c/em\u003e \u003cstrong\u003e9\u003c/strong\u003e, 1-12 (2019).\u003c/li\u003e\n\u003cli\u003eShams, S. et al. Deep generative breast cancer screening and diagnosis. In \u003cem\u003eInternational Conference on Medical Image Computing and Computer-Assisted Intervention\u003c/em\u003e, 859-867 (Springer, Cham 2018).\u003c/li\u003e\n\u003cli\u003eLi, H., Zhuang, S., Li, D. A., Zhao, J., \u0026amp; Ma, Y. Benign and malignant classification of mammogram images based on deep learning. \u003cem\u003eBiomedical Signal Processing and Control\u003c/em\u003e \u003cstrong\u003e51\u003c/strong\u003e, 347-354 (2019).\u003c/li\u003e\n\u003cli\u003eZhang, Q. et al. A novel algorithm for breast mass classification in digital mammography based on feature fusion. \u003cem\u003eJournal of Healthcare Engineering\u003c/em\u003e (2020).\u003c/li\u003e\n\u003cli\u003eMuramatsu, C. et al. Improving breast mass classification by shared data with domain transformation using a generative adversarial network. \u003cem\u003eComputers in biology and medicine\u003c/em\u003e \u003cstrong\u003e119\u003c/strong\u003e, 103698 (2020).\u003c/li\u003e\n\u003cli\u003eChakravarthy, S. S., \u0026amp; Rajaguru, H. Automatic Detection and Classification of Mammograms Using Improved Extreme Learning Machine with Deep Learning. \u003cem\u003eIRBM\u003c/em\u003e (2021).\u003c/li\u003e\n\u003cli\u003eKhan, H. N., Shahid, A. R., Raza, B., Dar, A. H., \u0026amp; Alquhayz, H. Multi-view feature fusion based four views model for mammogram classification using convolutional neural network. \u003cem\u003eIEEE Access\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 165724-165733 (2019).\u003c/li\u003e\n\u003cli\u003eKumar, I., Bhadauria, H. S., Virmani, J., \u0026amp; Thakur, S. A classification framework for prediction of breast density using an ensemble of neural network classifiers. \u003cem\u003eBiocybernetics and Biomedical Engineering\u003c/em\u003e \u003cstrong\u003e37\u003c/strong\u003e, 217-228 (2017).\u003c/li\u003e\n\u003cli\u003eAlkhaleefah, M. et al. Double-shot transfer learning for breast cancer classification from X-ray images. \u003cem\u003eApplied Sciences\u003c/em\u003e \u003cstrong\u003e10\u003c/strong\u003e, 3999 (2020).\u003c/li\u003e\n\u003cli\u003eFalcon\u0026iacute;, L., P\u0026eacute;rez, M., Aguilar, W., \u0026amp; Conci, A. Transfer Learning and Fine Tuning in Mammogram BI-RADS Classification. In \u003cem\u003e2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS)\u003c/em\u003e, 475-480 (IEEE, 2020).\u003c/li\u003e\n\u003cli\u003eMedeiros, A., Ohata, E. F., Silva, F. H., Rego, P. A., \u0026amp; Reboucas Filho, P. P. An approach to BI-RADS uncertainty levels classification via deep learning with transfer learning technique. In \u003cem\u003e2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS)\u003c/em\u003e, 603-608 (IEEE, 2020).\u003c/li\u003e\n\u003cli\u003eBi, W. L. et al. Artificial intelligence in cancer imaging: clinical challenges and applications. \u003cem\u003eCA: a cancer journal for clinicians\u003c/em\u003e \u003cstrong\u003e69\u003c/strong\u003e, 127-157 (2019).\u003c/li\u003e\n\u003cli\u003eTsochatzidis, L., Koutla, P., Costaridou, L., \u0026amp; Pratikakis, I. Integrating segmentation information into CNN for breast cancer diagnosis of mammographic masses. \u003cem\u003eComputer Methods and Programs in Biomedicine\u003c/em\u003e \u003cstrong\u003e200\u003c/strong\u003e, 105913 (2021).\u003c/li\u003e\n\u003cli\u003eLi, H., Chen, D., Nailon, W. H., Davies, M. E., \u0026amp; Laurenson, D. Dual Convolutional Neural Networks for Breast Mass Segmentation and Diagnosis in Mammography. Preprint at https://arxiv.org/abs/\u003cem\u003e2008.02957\u003c/em\u003e\u003cem\u003e \u003c/em\u003e(2020).\u003c/li\u003e\n\u003cli\u003eSarkar, P. R., Prabhakar, P., Mishra, D., \u0026amp; Subrahmanyam, G. Towards automated breast mass classification using deep learning framework. In \u003cem\u003e2019 IEEE International Conference on Data Science and Advanced Analytics (DSAA)\u003c/em\u003e, 453-462 (IEEE, 2019).\u003c/li\u003e\n\u003cli\u003eDhungel, N., Carneiro, G., \u0026amp; Bradley, A. P. Fully automated classification of mammograms using deep residual neural networks. In \u003cem\u003e2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017)\u003c/em\u003e, 310-314 (IEEE, 2017).\u003c/li\u003e\n\u003cli\u003eSingh, V. K. et al. Breast tumor segmentation and shape classification in mammograms using generative adversarial and convolutional neural network. \u003cem\u003eExpert Systems with Applications\u003c/em\u003e \u003cstrong\u003e139\u003c/strong\u003e, 112855 (2020).\u003c/li\u003e\n\u003cli\u003eAl-Antari, M. A., Al-Masni, M. A., Choi, M. T., Han, S. M., \u0026amp; Kim, T. S. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. \u003cem\u003eInternational journal of medical informatics\u003c/em\u003e \u003cstrong\u003e117\u003c/strong\u003e, 44-54 (2018).\u003c/li\u003e\n\u003cli\u003eAl-Antari, M. A., Al-Masni, M. A., \u0026amp; Kim, T. S. Deep learning computer-aided diagnosis for breast lesion in digital mammogram. \u003cem\u003eDeep Learning in Medical Image Analysis\u003c/em\u003e, 59-72 (2020).\u003c/li\u003e\n\u003cli\u003eBaccouche, A., Garcia-Zapirain, B., Castillo Olea, C., \u0026amp; Elmaghraby, A. S. Connected-UNets: a deep learning architecture for breast mass segmentation. \u003cem\u003eNPJ Breast Cancer\u003c/em\u003e \u003cstrong\u003e7\u003c/strong\u003e, 1-12 (2021).\u003c/li\u003e\n\u003cli\u003eM\u0026uuml;ller, R., Kornblith, S., \u0026amp; Hinton, G. When does label smoothing help?. Preprint at https://arxiv.org/abs/\u003cem\u003e1906.02629\u003c/em\u003e (2019).\u003c/li\u003e\n\u003cli\u003eHe, K., Zhang, X., Ren, S., \u0026amp; Sun, J. Deep residual learning for image recognition. In \u003cem\u003eProceedings of the IEEE conference on computer vision and pattern recognition\u003c/em\u003e, 770-778 (2016)\u003c/li\u003e\n\u003cli\u003eGarcia-Gasulla, D. et al. On the behavior of convolutional nets for feature extraction. \u003cem\u003eJournal of Artificial Intelligence Research\u003c/em\u003e \u003cstrong\u003e61\u003c/strong\u003e, 563-592 (2018).\u003c/li\u003e\n\u003cli\u003eSzegedy, C., Ioffe, S., Vanhoucke, V., \u0026amp; Alemi, A. A. Inception-v4, inception-resnet and the impact of residual connections on learning. In \u003cem\u003eThirty-first AAAI conference on artificial intelligence\u003c/em\u003e (2017).\u003c/li\u003e\n\u003cli\u003eYu, X. et al. ResNet-SCDA-50 for breast abnormality classification. \u003cem\u003eIEEE/ACM transactions on computational biology and bioinformatics\u003c/em\u003e \u003cstrong\u003e18\u003c/strong\u003e, 94-102 (2020).\u003c/li\u003e\n\u003cli\u003eHe, K., Zhang, X., Ren, S., \u0026amp; Sun, J. Identity mappings in deep residual networks. In \u003cem\u003eEuropean conference on computer vision\u003c/em\u003e, 630-645 (Springer, Cham 2016).\u003c/li\u003e\n\u003cli\u003eChen, Y. et al. Fine-tuning ResNet for breast cancer classification from mammography. In \u003cem\u003eThe International Conference on Healthcare Science and Engineering\u003c/em\u003e, 83-96 (Springer, Singapore 2018).\u003c/li\u003e\n\u003cli\u003eBellmann, P., Thiam, P., \u0026amp; Schwenker, F. Multi-classifier-systems: architectures, algorithms and applications. In \u003cem\u003eComputational Intelligence for Pattern Recognition\u003c/em\u003e, 83-113 (Springer, Cham 2018).\u003c/li\u003e\n\u003cli\u003eLee, R. S. et al. A curated mammography data set for use in computer-aided detection and diagnosis research. \u003cem\u003eScientific data\u003c/em\u003e \u003cstrong\u003e4\u003c/strong\u003e, 1-9 (2017).\u003c/li\u003e\n\u003cli\u003eMoreira, I. C. et al. Inbreast: toward a full-field digital mammographic database. \u003cem\u003eAcademic radiology\u003c/em\u003e \u003cstrong\u003e19\u003c/strong\u003e, 236-248 (2012).\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Mass classification, BI-RADS, Shape, Stacked ensemble, Computer-aided diagnosis (CAD)","lastPublishedDoi":"10.21203/rs.3.rs-1389924/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-1389924/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eA computer-aided diagnosis (CAD) system requires automated stages of tumor detection, segmentation, and classification that are integrated sequentially in one framework to assist the radiologists with a final diagnosis decision. In this paper, we introduce the final step of breast mass classification and diagnosis using a stacked ensemble of residual neural network (ResNet) models (i.e. ResNet50V2, ResNet101V2, and ResNet152V2). The work presents the task of classifying the detected and segmented breast masses into malignant or benign, and diagnosing the Breast Imaging Reporting and Data System (BI-RADS) assessment category with a score from 2 to 6 and the shape as oval, round, lobulated, or irregular. The proposed methodology was evaluated on two publicly available datasets, the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) and INbreast, and additionally on a private dataset. Comparative experiments were conducted on the individual models and an average ensemble of models with an XGBoost classifier. Qualitative and quantitative results show that the proposed model achieved better performance for 1) Pathology classification with an accuracy of 95.13%, 99.20%, and 95.88%; 2) BI-RADS category classification with an accuracy of 85.38%, 99%, and 96.08% respectively on CBIS-DDSM, INbreast, and the private dataset; and 3) shape classification with 90.02% on the CBIS-DDSM dataset. Our results demonstrate that our proposed integrated framework could benefit from all automated stages to outperform the latest deep learning methodologies.\u003c/p\u003e","manuscriptTitle":"An integrated Framework for Breast Mass Classification and Diagnosis using Stacked Ensemble of Residual Neural Networks","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2022-03-01 21:34:55","doi":"10.21203/rs.3.rs-1389924/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Major revision","date":"2022-05-30T13:31:51+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2022-05-23T10:53:06+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"1df6224e-3e65-467c-b982-f13714aa10d6","date":"2022-05-19T13:54:15+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2022-05-19T12:25:05+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2022-03-01T21:32:49+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2022-02-26T07:15:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2022-02-26T07:09:33+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2022-02-23T18:42:35+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8bf97385-4435-4b54-9c58-e2830c8398bc","owner":[],"postedDate":"March 1st, 2022","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2022-06-27T10:14:28+00:00","versionOfRecord":[],"versionCreatedAt":"2022-03-01 21:34:55","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-1389924","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-1389924","identity":"rs-1389924","version":["v1"]},"buildId":"7rjqhiLT3MXkJMwkYKINL","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}