Optimizing Deep Learning for Skin Cancer: A Comparative Study of Convolutional and Attention-Based Models

Optimizing Deep Learning for Skin Cancer: A Comparative Study of Convolutional and Attention-Based Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Optimizing Deep Learning for Skin Cancer: A Comparative Study of Convolutional and Attention-Based Models Khaled Wael Ezzat This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9285260/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Skin cancer is among the most prevalent malignancies worldwide, with over 1.5 million new cases estimated in 2022 alone according to GLOBOCAN data [ 1 ] . Despite the availability of dermoscopy, experienced dermatologists achieve a melanoma detection sensitivity of approximately 75–84% using visual examination, a rate that underscores the diagnostic limitations of unaided clinical assessment [ 2 ] . This study presents a systematic comparison of five deep learning architectures for the automated classification of seven skin lesion types using the HAM10000 dataset [ 6 ] , comprising 10,015 dermoscopic images. We evaluate four architectures spanning both convolutional and attention-based paradigms: ResNet-50, EfficientNet-B4, ConvNeXt-Base, Swin Transformer-Base, and Vision Transformer (ViT-B/16). To address the pronounced class imbalance inherent in the dataset, we employed patient-level data partitioning via GroupShuffleSplit to prevent lesion leakage across splits, and WeightedRandomSampler during training. All models were trained using AdamW optimization with label smoothing and mixed-precision training. Transformer-based architectures were further stabilized through linear warmup scheduling and stochastic depth regularization. Our best single model, ViT-B/16, achieved a test accuracy of 85.66% and a macro AUC-ROC of 0.9629. An ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the highest overall performance with a test accuracy of 86.57%, a balanced accuracy of 79.98%, a macro F1-score of 0.7856, and a macro AUC-ROC of 0.9811. These results demonstrate that heterogeneous ensemble strategies combining architecturally diverse models offer a meaningful improvement over individual classifiers in dermoscopic lesion classification. Artificial Intelligence and Machine Learning Multi-Class Skin Lesion Classification HAM10000 Benchmark Swin Transformer-Base Heterogeneous Ensemble Learning Patient-Level Data Partitioning GroupShuffleSplit Class Imbalance Mitigation Stochastic Depth Regularization Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Figure 9 1. Introduction Skin cancers are the most commonly diagnosed group of cancers worldwide, with more than 1.5 million new cases estimated in 2022 [ 1 ] . Among these, melanoma represents a disproportionate share of mortality; if current incidence rates continue, the global burden of melanoma is projected to reach 510,000 new cases and 96,000 deaths annually by 2040, representing a roughly 50% increase in incidence from 2020 figures [ 3 ] . The clinical consequences of delayed diagnosis are severe, as early-stage intervention substantially improves prognosis and reduces the need for aggressive surgical treatment [ 3 ] . Current standard-of-care diagnosis relies on visual inspection augmented by dermoscopy. However, this process is both time-intensive and subject to inter-observer variability. Even with the combination of visual examination and dermoscopic imaging, the absolute diagnostic accuracy for melanoma detection among experienced dermatologists ranges from 75 to 84% [ 2 ] . This ceiling reflects not only the inherent difficulty of distinguishing morphologically similar lesion subtypes, but also systemic constraints including limited specialist availability, particularly in underserved and rural regions [ 4 ] . The application of deep convolutional neural networks to dermoscopic image analysis has demonstrated substantial promise in bridging this diagnostic gap. Transfer learning from large-scale image classification benchmarks such as ImageNet has emerged as the dominant paradigm, allowing models pretrained on millions of natural images to be adapted to medical imaging tasks with comparatively small domain-specific datasets [ 5 ] . Tschandl et al. introduced the HAM10000 dataset as a large-scale, multi-source collection of dermoscopic images spanning seven clinically relevant lesion categories, providing a standardized benchmark for evaluating such approaches [ 6 ] . Prior work in this domain has focused predominantly on CNN-based architectures. Lin and Ghanta [ 7 ] compared MobileNetV2, ResNet50V2, EfficientNetV2B0, and VGG16 on the ISIC dataset, finding VGG16 to achieve the highest test accuracy of 84.7% and an AUC of 0.95 in a binary classification setting. However, the emergence of attention-based Vision Transformers [ 8 ] and hybrid architectures such as the Swin Transformer [ 9 ] has introduced a fundamentally different inductive bias — one that models long-range spatial dependencies through self-attention rather than local convolutional filters. Whether these architectural differences translate into measurable performance gains in the specific context of dermoscopic classification, and how such models interact in ensemble settings, remains an open question. This study addresses that question directly. We make the following contributions: (1) a rigorous, patient-level comparison of five architectures spanning CNN and Transformer families on the 7-class HAM10000 benchmark; (2) an analysis of architecture-specific training considerations, including the impact of learning rate scheduling on Transformer fine-tuning stability; (3) an ensemble strategy combining a CNN-based and a Transformer-based model, optimized via validation-set weight search; and (4) explainability visualizations through Grad-CAM and attention maps that illuminate the spatial reasoning of each model family. 2. Related Work 2.1 CNN-Based Approaches on HAM10000 The HAM10000 dataset, introduced by Tschandl et al. [ 6 ] , rapidly became the dominant benchmark for multi-class dermoscopic lesion classification. Early transfer learning studies demonstrated that pretrained CNN backbones could be effectively adapted to this domain. Rezvantalab et al. applied DenseNet201, ResNet152, InceptionV3, and InceptionResNetV2 to the HAM10000 benchmark, with DenseNet201 achieving an accuracy of 86.59% [ 10 ] . Garg et al. combined VGG16 and ResNet50 with ensemble metalearners including random forest, XGBoost, and SVM, reporting an accuracy of 90.51% on the same seven-class task [ 9 ] . Lin and Ghanta [ 7 ] further benchmarked four CNN architectures in a binary classification setting, with VGG16 achieving the highest AUC of 0.95. These studies collectively established the feasibility of transfer learning for dermoscopic classification, yet remained confined to the CNN paradigm and, in several cases, applied random rather than patient-level data splitting, potentially inflating reported metrics through lesion leakage. 2.2 Transformer-Based Approaches The introduction of the Vision Transformer (ViT) by Dosovitskiy et al. [ 8 ] , which models image patches as sequential tokens processed through multi-head self-attention, opened a new research direction for medical image analysis. A systematic review of transformer applications in skin lesion classification identified 57 relevant studies published between 2017 and 2023, noting that HAM10000 was the most widely used benchmark across this body of work [ 11 ] . Ayas [ 15 ] proposed a dedicated Swin Transformer model for multi-class skin lesion classification, demonstrating that the hierarchical shifted-window attention mechanism translates effectively to dermoscopic feature extraction. Subsequent work proposed SkinSwinViT, integrating Swin Transformer with a global attention mechanism, and demonstrated superior performance compared to prior CNN-based approaches [ 12 ] . Despite this progress, direct comparative evaluation of CNN and Transformer families under controlled, patient-level experimental conditions on the full seven-class HAM10000 task remains limited. 2.3 Ensemble Methods Ensemble learning has consistently yielded performance gains in skin lesion classification by exploiting the complementary feature representations of constituent models. Liu et al. developed SkinNet using stacking ensemble methods, reporting an accuracy of 86.7% and an AUC of 0.96 on HAM10000 [ 13 ] . A more recent study introduced an ensemble framework integrating Swin Transformer, ViT, and EfficientNet-B4, achieving 98.5% accuracy on a custom clinical dataset, with the authors explicitly attributing the performance gain to the complementary inductive biases of CNN and Transformer architectures [ 18 ] . EF-SwinNet, a hybrid model combining EfficientNet and Swin Transformer, further demonstrated that CNN-Transformer fusion addresses the limitation of CNNs in capturing long-range spatial dependencies while retaining parameter efficiency [ 19 ] . The present study builds directly on this insight, evaluating a post-hoc weighted ensemble and searching for optimal combination weights on the validation set. 3. Methodology 3.1 Dataset and Patient-Level Splitting All experiments in this study use the HAM10000 dataset [ 6 ] , comprising 10,015 dermoscopic images across seven lesion classes: Melanocytic Nevi (NV), Melanoma (MEL), Benign Keratosis (BKL), Basal Cell Carcinoma (BCC), Actinic Keratoses (AKIEC), Vascular Lesions (VASC), and Dermatofibroma (DF). The dataset exhibits pronounced class imbalance, with Melanocytic Nevi accounting for nearly 67% of all samples. A critical methodological consideration in skin lesion datasets is the presence of multiple images corresponding to the same lesion, which can introduce data leakage if splitting is performed at the image level. To prevent this, we applied patient-level partitioning using GroupShuffleSplit on the lesion_id field, ensuring that all images of a given lesion appear exclusively within one split. The resulting distribution allocates 72% of images to training, 13% to validation, and 15% to testing. We verified the absence of lesion-level overlap across all three splits via set intersection checks. 3.2 Class Imbalance Handling To address the severe class imbalance, we employed WeightedRandomSampler during training, assigning each training sample a weight inversely proportional to its class frequency. This ensures that minority classes such as Dermatofibroma and Vascular Lesions are oversampled to appear with frequency comparable to the dominant Melanocytic Nevi class during each training epoch, without modifying the underlying dataset distribution. 3.3 Image Preprocessing and Augmentation All images were resized to the native input resolution of each architecture. CNN-based models and Swin Transformer-Base were trained at 224×224 pixels, while EfficientNet-B4 was trained at its recommended resolution of 380×380 pixels. Pixel values were normalized using ImageNet mean and standard deviation across all models. Training-time augmentation was restricted to geometric transformations — horizontal flip, vertical flip, 90-degree rotation, and moderate shift-scale-rotate — to avoid introducing artificial color or texture variations that might confound lesion appearance. 3.4 Model Architectures We evaluate five pretrained architectures spanning two architectural paradigms. ResNet-50 [11r] serves as the baseline CNN, comprising 50 layers with residual connections that address the vanishing gradient problem (~ 25M parameters). EfficientNet-B4 [12r] applies compound scaling across depth, width, and resolution (~ 19M parameters, trained at 380×380). ConvNeXt-Base [13r] modernizes the CNN design by incorporating depthwise convolutions and LayerNorm while retaining convolutional inductive bias (~ 89M parameters). Swin Transformer-Base [ 9 ] partitions input images into non-overlapping patches and processes them through shifted window attention, enabling hierarchical feature extraction (~ 88M parameters). ViT-B/16 [ 8 ] applies pure multi-head self-attention across a sequence of 196 non-overlapping 16×16 patches (~ 86M parameters). 3.5 Training Configuration All models were trained using AdamW optimization [ 14 ] with a weight decay of 1×10⁻². The loss function was cross-entropy with label smoothing of 0.1 [ 15 ] , which penalizes overconfident predictions and improves calibration. Mixed-precision training (fp16) was applied to reduce memory consumption and training time. Gradient clipping with a maximum norm of 1.0 was applied at each update step. CNN-based models were trained with CosineAnnealingWarmRestarts scheduling [ 16 ] with T₀=10 and T_mult = 2. Transformer-based models required a modified scheduling strategy: a linear warmup phase gradually increases the learning rate from zero to its target value before transitioning to cosine decay without restarts. This modification is motivated by the sensitivity of attention weight initialization to large gradient updates in the early training phase, as discussed in the original Swin Transformer paper [ 9 ] . Swin Transformer-Base additionally uses stochastic depth regularization (drop_path_rate = 0.3) [ 17 ] , and ViT-B/16 uses both stochastic depth (drop_path_rate = 0.1) and dropout (rate = 0.1). Early stopping was applied with a patience of 7–8 epochs. 3.6 Ensemble Strategy The ensemble combines the softmax probability outputs of EfficientNet-B4 and Swin Transformer-Base through weighted averaging. The combination weight was optimized via a coarse grid search over the range [0.0, 1.0] in steps of 0.1, evaluated on the validation set. The resulting optimal weights of w = 0.4 for EfficientNet-B4 and w = 0.6 for Swin Transformer-Base were applied to the test set for final evaluation. 3.7 Evaluation Metrics Model performance is reported across four metrics: test accuracy, balanced accuracy (mean per-class recall), macro-averaged F1-score, and macro-averaged AUC-ROC. The latter three metrics provide class-imbalance-robust evaluation, equally weighting all seven lesion categories regardless of their prevalence in the test set. 4. Results 4.1 Individual Model Performance Table 1 summarizes the test set performance of all five models across the four evaluation metrics. ResNet-50, serving as the CNN baseline, achieved a test accuracy of 75.90% and a macro F1-score of 0.6681, establishing a lower reference bound. EfficientNet-B4 improved substantially over this baseline, reaching 85.07% test accuracy and a macro AUC-ROC of 0.9712. ConvNeXt-Base achieved a comparable test accuracy of 84.74%, with notably stronger balanced accuracy (76.34%) and macro F1-score (0.7510), suggesting superior handling of the minority classes. Among Transformer-based architectures, Swin Transformer-Base achieved the highest balanced accuracy (79.66%) and AUC-ROC (0.9717) of any individual model. ViT-B/16 attained the highest single-model test accuracy at 85.66%, with a macro F1-score of 0.7741. Table 1 Test set performance of all evaluated models on HAM10000. Model Test Acc. Balanced Acc. Macro F1 AUC-ROC ResNet-50 (Baseline) 0.7590 0.7500 0.6681 0.9548 EfficientNet-B4 0.8507 0.7083 0.7380 0.9712 ConvNeXt-Base 0.8474 0.7634 0.7510 0.9428 Swin Transformer-Base 0.8507 0.7966 0.7719 0.9717 ViT-B/16 0.8566 0.7720 0.7741 0.9629 Ensemble (EfficientNet-B4 + Swin) 0.8657 0.7998 0.7856 0.9811 4.2 Training Dynamics A consistent pattern of overfitting was observed across all models, with training accuracy converging to values between 97% and 99% while validation accuracy plateaued at substantially lower levels, reflecting a generalization gap of approximately 13–14 percentage points. This gap is consistent with the limited scale of HAM10000 relative to the capacity of the evaluated architectures, and aligns with findings reported in prior work on the same benchmark [ 10 ] . Transformer-based models exhibited markedly different training dynamics compared to CNNs. When trained with CosineAnnealingWarmRestarts scheduling, Swin Transformer-Base displayed severe validation accuracy oscillations at each learning rate restart. This instability was eliminated upon replacing the restart-based scheduler with linear warmup followed by smooth cosine decay, as prescribed in the original Swin Transformer publication [ 9 ] . ViT-B/16 required an extended warmup period of five epochs and a lower peak learning rate of 3×10⁻⁵ to achieve stable convergence. 4.3 Ensemble Optimization and Final Performance The ensemble weight search on the validation set identified w = 0.4 for EfficientNet-B4 and w = 0.6 for Swin Transformer-Base as optimal. The ensemble achieved a test accuracy of 86.57%, a balanced accuracy of 79.98%, a macro F1-score of 0.7856, and a macro AUC-ROC of 0.9811, outperforming all individual models across every reported metric. 4.4 Explainability Grad-CAM visualizations for CNN-based models confirmed that activation patterns were spatially concentrated on lesion boundaries and internal texture regions. Attention maps generated for Swin Transformer-Base revealed broader spatial coverage distributed across the full lesion area. ViT-B/16 attention maps showed similar global coverage, attributable to the unrestricted receptive field of pure self-attention from the first layer. These qualitative differences in spatial reasoning are consistent with the quantitative finding that Transformer-based models achieve stronger balanced accuracy. 5. Discussion 5.1 CNN vs. Transformer Performance The results present a nuanced picture of the relative merits of convolutional and attention-based architectures. On overall test accuracy, ViT-B/16 narrowly outperforms all other individual models. However, on balanced accuracy and AUC-ROC — metrics that weight minority class performance more equitably — Swin Transformer-Base is the strongest individual model, achieving 79.66% balanced accuracy compared to 77.20% for ViT-B/16 and 70.83% for EfficientNet-B4. This divergence between accuracy and balanced accuracy is clinically significant. In a seven-class dermoscopic classification task where minority classes include malignant conditions such as Basal Cell Carcinoma and Actinic Keratoses, a model that achieves high overall accuracy through strong performance on the dominant Melanocytic Nevi class while underperforming on rarer malignancies is of limited clinical utility. The consistently stronger balanced accuracy of Transformer-based architectures suggests that attention-based models may be inherently better suited to imbalanced medical classification tasks by virtue of their capacity to model global image context without the locality bias imposed by convolutional kernels. 5.2 The Role of Scheduling in Transformer Fine-Tuning A central finding of this study is that the choice of learning rate scheduler is not a secondary implementation detail but a primary determinant of whether Transformer fine-tuning converges stably. The CosineAnnealingWarmRestarts schedule, which is effective for CNN-based models, caused repeated destabilization of Swin Transformer training due to the hard learning rate resets that it imposes every T₀ epochs. Replacing the restart schedule with a smooth linear warmup followed by monotone cosine decay, as recommended in Liu et al. [ 9 ] , eliminated this instability entirely. This finding has practical implications for future work adapting Transformer architectures to small-scale medical datasets, and underscores the importance of architecture-specific training protocols rather than uniform hyperparameter application across model families. 5.3 Ensemble Gains and Architectural Complementarity The ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the strongest performance across all four reported metrics simultaneously, with an AUC-ROC of 0.9811 representing a gain of 0.0094 over the best individual model. These gains are consistent and appear across all metrics, suggesting that the combination captures genuine complementarity between the two model families. EfficientNet-B4 activations are concentrated on local texture and edge features, while Swin Transformer-Base attention spans the full lesion region. When combined through probability averaging, the ensemble benefits from both precise local feature discrimination and holistic spatial context modeling, consistent with findings reported in related ensemble studies [ 18 , 19 ] . 5.4 Limitations Several limitations merit acknowledgment. First, all experiments are conducted on a single dataset, and generalization to clinical deployment would require external validation on independent patient cohorts. Second, the limited scale of HAM10000 constrains the full expression of large Transformer capacity and is likely the primary driver of the generalization gap observed across all models. Third, the ensemble strategy is restricted to simple weighted averaging; more sophisticated combination methods may yield additional gains. Finally, the explainability visualizations provided are qualitative rather than quantitative, and rigorous evaluation against radiologist-annotated saliency maps would be necessary to draw definitive conclusions regarding clinical interpretability. 6. Conclusion This study presented a systematic evaluation of five deep learning architectures for the automated classification of seven skin lesion types using the HAM10000 dermoscopic benchmark. By comparing CNN-based models — ResNet-50, EfficientNet-B4, and ConvNeXt-Base — against attention-based Transformer architectures — Swin Transformer-Base and ViT-B/16 — under controlled, patient-level experimental conditions, this work provides a principled assessment of the relative strengths and limitations of each architectural family for this clinically important task. The results demonstrate that Transformer-based architectures achieve competitive or superior performance compared to state-of-the-art CNNs, with Swin Transformer-Base attaining the strongest balanced accuracy and AUC-ROC among individual models, and ViT-B/16 achieving the highest overall test accuracy. A key practical finding is that architecture-specific training protocols are essential for Transformer fine-tuning: the linear warmup followed by smooth cosine decay is necessary for stable convergence, in contrast to the restart-based scheduling effective for CNNs. The ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the strongest performance across all four evaluated metrics simultaneously — test accuracy of 86.57%, balanced accuracy of 79.98%, macro F1-score of 0.7856, and macro AUC-ROC of 0.9811 — consistent with the architectural complementarity between local convolutional feature extraction and global attention-based spatial reasoning. Future work should prioritize external validation on independent clinical cohorts, investigation of more sophisticated ensemble combination strategies, and exploration of data augmentation techniques specifically designed to address the class imbalance and limited scale of dermoscopic datasets. The integration of patient metadata alongside image features represents an additional avenue with demonstrated potential for performance improvement in this domain [19m] . References Ferlay J, Ervik M, Lam F et al (2024) Global Cancer Observatory: Cancer Today. Lyon: International Agency for Research on Cancer; Available from: https://gco.iarc.who.int/today Saleh GM, Litvinova M, Lakhiani DD et al (2020) Machine learning and deep learning methods for skin lesion classification and diagnosis: A systematic review. Diagnostics 11:1390 Arnold M, Singh D, Laversanne M et al (2022) Global burden of cutaneous melanoma in 2020 and projections to 2040. JAMA Dermatol 158(5):495–503 Brinker TJ, Hekler A, Utikal JS et al (2018) Skin cancer classification using convolutional neural networks: systematic review. J Med Internet Res 20(10):e11936 Kim HE, Cosa-Linan A, Santhanam N et al (2022) Transfer learning for medical image classification: a literature review. BMC Med Imaging 22(1):69 Tschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161 Lin R, Ghanta S (2024) Transfer learning with convolutional neural network-based models for skin cancer classification. J Emerg Investig. ;7 Dosovitskiy A, Beyer L, Kolesnikov A et al (2021) An image is worth 16x16 words: Transformers for image recognition at scale. ICLR Liu Z, Lin Y, Cao Y et al (2021) Swin Transformer: Hierarchical vision transformer using shifted windows. ICCV Himel GMS, Islam MM, Al-Aff KA et al (2024) Skin cancer segmentation and classification using Vision Transformer for automatic analysis in dermatoscopy-based noninvasive digital system. Int J Biomed Imaging Mishra NK, Bhatt C, Biswas M et al (2024) Transformers in skin lesion classification and diagnosis: a systematic review. medRxiv Chen X, Li C, Zhang Y et al (2024) SkinSwinViT: a lightweight transformer-based method for multiclass skin lesion classification with enhanced generalization capabilities. Appl Sci 14(10):4005 Liu X, Yu Z, Tan L et al (2024) Enhancing skin lesion diagnosis with ensemble learning. Proceedings of the 4th International Conference on Computer Science and Blockchain (CCSB) Loshchilov I, Hutter F (2019) Decoupled weight decay regularization. ICLR Szegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. CVPR Loshchilov I, Hutter F (2017) SGDR: Stochastic gradient descent with warm restarts. ICLR Huang G, Sun Y, Liu Z et al (2016) Deep networks with stochastic depth. ECCV Rasheed J, Jan B, Taher F et al (2025) Deep ensemble learning for multiclass skin lesion classification. Bioengineering 12(9):934 Ashraf YM et al (2024) EF-SwinNet: a hybrid EfficientNet-Swin Transformer model for skin lesion classification on HAM10000. ICRPSET Additional Declarations The authors declare no competing interests. Supplementary Files ModelsResults.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9285260","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":615580081,"identity":"fec163b0-fb36-4639-913c-e7403d613dd4","order_by":0,"name":"Khaled Wael Ezzat","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA40lEQVRIiWNgGAWjYBACCRDxoOJ/fWN7A5BlYEGkloQzzIzNPQdAWiSI1JLYwszYPiMBzscPJGckP3yQ2MDGzDvz+dUNPwokGPjbuxPwapGWSDM2SNzBwyY5O6fsZg/QYRJnzm7Aq0VOOsFMIvGMBI/h7Jy0GzxALQYSuYS0pH//kdhmIGF/80zazT/EaJGWzjFjSGxLMGCcwX7sNlG2SM5/UyyRcOZAAmNPDtttGQMJHoJ+kThzfOOHDxVALe3Hn91888dGjr+9F78WJMBjACaJVQ4C7A9IUT0KRsEoGAUjCAAAIJlJ+UD/F2wAAAAASUVORK5CYII=","orcid":"","institution":"Fayoum STEM School","correspondingAuthor":true,"prefix":"","firstName":"Khaled","middleName":"Wael","lastName":"Ezzat","suffix":""}],"badges":[],"createdAt":"2026-04-01 01:50:48","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-9285260/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9285260/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106381493,"identity":"fe09213b-f1e7-451b-8acd-9d8d3bb0a0e6","added_by":"auto","created_at":"2026-04-08 05:22:30","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1028210,"visible":true,"origin":"","legend":"\u003cp\u003eThe methodology consists of four primary phases:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e1. \u003cstrong\u003eData Partitioning:\u003c/strong\u003e Utilizing the HAM10000 dataset , patient-level splitting is performed via \u003cstrong\u003eGroupShuffleSplit\u003c/strong\u003e to ensure zero lesion leakage between training, validation, and test sets.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e2. \u003cstrong\u003eTraining Preparation:\u003c/strong\u003e To handle severe class imbalance , a \u003cstrong\u003eWeighted Random Sampler\u003c/strong\u003e is integrated alongside standard data augmentation.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e3. \u003cstrong\u003eModel Training:\u003c/strong\u003e Five distinct architectures (ResNet-50, EfficientNet-B4, ConvNeXt-Base, Swin Transformer-Base, and ViT-B/16) are evaluated. 4. \u003cstrong\u003eEnsemble Strategy:\u003c/strong\u003e A heterogeneous ensemble is formed by combining the top-performing CNN (EfficientNet-B4) and Transformer (Swin Transformer-Base) models through weighted probability averaging, optimized via validation-set grid search\u003c/p\u003e","description":"","filename":"systemarchitecture.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/3494c2f5eaa2ade0fcac4b13.png"},{"id":106403923,"identity":"7539d691-98f9-4ffe-91ef-f91fe22d6223","added_by":"auto","created_at":"2026-04-08 09:15:13","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":67600,"visible":true,"origin":"","legend":"\u003cp\u003eA comprehensive benchmark of the five individual models against the proposed ensemble. (Left to Right): \u003cstrong\u003eTest Accuracy, Balanced Accuracy, Macro F1-score, and AUC-ROC\u003c/strong\u003e. The \u003cstrong\u003eEnsemble model\u003c/strong\u003e (red bar) consistently outperforms individual classifiers across all metrics, achieving a peak \u003cstrong\u003eAUC-ROC of 0.9811\u003c/strong\u003e and \u003cstrong\u003e86.57% test accuracy\u003c/strong\u003e. The comparison highlights that while \u003cstrong\u003eViT-B/16\u003c/strong\u003e is the strongest single model for accuracy, \u003cstrong\u003eSwin Transformer-Base\u003c/strong\u003e offers superior balanced performance for minority classes.\u003c/p\u003e","description":"","filename":"resultssummary.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/1da7e5d59aec3306fe49264f.png"},{"id":106414810,"identity":"ee1c8d49-35c4-4bbf-84be-fd1f3979ec5b","added_by":"auto","created_at":"2026-04-08 10:24:59","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":90667,"visible":true,"origin":"","legend":"\u003cp\u003eThe matrix visualizes the classification performance of the \u003cstrong\u003eEfficientNet-B4 + Swin Transformer\u003c/strong\u003e ensemble on the test set. (Left) Absolute prediction counts; (Right) Normalized recall per class. The ensemble demonstrates high diagnostic precision for dominant classes like \u003cstrong\u003eMelanocytic Nevi (91%)\u003c/strong\u003e and \u003cstrong\u003eBasal Cell Carcinoma (91%)\u003c/strong\u003e. Importantly, it maintains robust performance on critical minority classes such as \u003cstrong\u003eVascular Lesions (83%)\u003c/strong\u003e and \u003cstrong\u003eMelanoma (72%)\u003c/strong\u003e, validating the effectiveness of the \u003cstrong\u003eWeighted Random Sampler\u003c/strong\u003e in mitigating class imbalance.\u003c/p\u003e","description":"","filename":"ensembelmodelconfusionmatrix.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/05466bd3636ff2e128bf212c.png"},{"id":106414979,"identity":"deee524b-4b00-4547-8e8b-ab4196ea0844","added_by":"auto","created_at":"2026-04-08 10:31:31","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":3129050,"visible":true,"origin":"","legend":"\u003cp\u003eThis figure displays Gradient-weighted Class Activation Mapping (Grad-CAM) across representative samples of different lesion types. The heatmaps confirm that the CNN-based model primarily focuses on \u003cstrong\u003elocal texture features\u003c/strong\u003e and \u003cstrong\u003elesion boundaries\u003c/strong\u003e to make its predictions. These localized activation patterns complement the broader attention maps of Transformer-based models, providing a physiological basis for the performance gains observed in the ensemble strategy.\u003c/p\u003e","description":"","filename":"GRADCAMexample.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/cfcdffc3fcbc0e77757845cb.png"},{"id":106403953,"identity":"28df7d58-ac64-45b3-b20f-892f93d9bf23","added_by":"auto","created_at":"2026-04-08 09:15:16","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":76325,"visible":true,"origin":"","legend":"\u003cp\u003eThe plots illustrate the training progression over 22 epochs:\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* \u003cstrong\u003eLoss (Left):\u003c/strong\u003e Displays the cross-entropy loss with label smoothing. The steady decline in training loss versus the plateauing validation loss indicates the model reaching its capacity within the HAM10000 dataset scale.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* \u003cstrong\u003eAccuracy (Center):\u003c/strong\u003e Shows the evolution of training and validation accuracy. A generalization gap of approximately 13-14% is observed, consistent with the limited scale of the dataset relative to the Transformer’s high parameter count.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e* \u003cstrong\u003eLearning Rate (Right):\u003c/strong\u003e Visualizes the \u003cstrong\u003eLinear Warmup\u003c/strong\u003e followed by a \u003cstrong\u003eCosine Decay\u003c/strong\u003e schedule. This specific strategy was essential to stabilize the attention mechanism and prevent the validation oscillations observed with standard restart-based schedulers.\u003c/p\u003e","description":"","filename":"SwinTransformerROCCurve.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/852cc60fb3f6e59cbcb91d07.png"},{"id":106404532,"identity":"c5261e12-fd71-4487-aea7-8f853e4e1fed","added_by":"auto","created_at":"2026-04-08 09:16:11","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":73268,"visible":true,"origin":"","legend":"\u003cp\u003eTraining progression for the pure attention-based ViT-B/16 model. As shown in the learning rate plot (Right), a \u003cstrong\u003eLinear Warmup\u003c/strong\u003e phase was crucial for stabilizing the self-attention mechanisms before transitioning to cosine decay. This strategy enabled the model to achieve the \u003cstrong\u003ehighest single-model test accuracy of 85.66%\u003c/strong\u003e. The narrow fluctuations in validation accuracy (Center) validate the effectiveness of the specialized Transformer training protocol used in this study.\u003c/p\u003e","description":"","filename":"VitB16ROCCurve.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/8a820f9590ca7fdbf9d88d53.png"},{"id":106404517,"identity":"69b0202e-18a8-47e0-80ba-8d9cfa097a57","added_by":"auto","created_at":"2026-04-08 09:16:09","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":81801,"visible":true,"origin":"","legend":"\u003cp\u003eThe plots illustrate the training trajectory over 30 epochs for the EfficientNet-B4 architecture. (Left) Cross-entropy loss; (Center) Accuracy; (Right) Learning rate schedule using \u003cstrong\u003eCosineAnnealingWarmRestarts\u003c/strong\u003e. The model achieves a test accuracy of 85.07%. Notably, the learning rate resets (Right) facilitate the exploration of the loss landscape, helping the model escape local minima during the fine-tuning process on the HAM10000 benchmark.\u003c/p\u003e","description":"","filename":"EfficientNetB4ROCCurve.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/7b4d0c6013778e8f561c505b.png"},{"id":106381498,"identity":"de148b83-cf0f-4913-be8b-a8d070f2b3ec","added_by":"auto","created_at":"2026-04-08 05:22:30","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":87429,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance metrics for ConvNeXt-Base, which modernizes the convolutional paradigm. Despite having the largest parameter count among the evaluated CNNs (~89M), the model shows stable convergence. The \u003cstrong\u003eBalanced Accuracy (76.34%)\u003c/strong\u003e and \u003cstrong\u003eMacro F1-score (0.7510)\u003c/strong\u003e indicate that this architecture is particularly effective at recognizing minority lesion classes compared to traditional CNN baselines like ResNet-50.\u003c/p\u003e","description":"","filename":"ConvNexROCCurve.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/7768650daf8f4ff85aef04a3.png"},{"id":106381503,"identity":"ceef040c-5562-43e0-977c-07cb9439d39e","added_by":"auto","created_at":"2026-04-08 05:22:30","extension":"png","order_by":9,"title":"Figure 9","display":"","copyAsset":false,"role":"figure","size":78256,"visible":true,"origin":"","legend":"\u003cp\u003ePerformance curves for the CNN baseline over 25 epochs. The \u003cstrong\u003eLearning Rate plot\u003c/strong\u003e (right) illustrates the \u003cstrong\u003eCosineAnnealingWarmRestarts\u003c/strong\u003e schedule, which uses periodic resets to escape local minima during fine-tuning. Despite stable convergence in training loss, the observed generalization gap (approx. 13-14%) is consistent with the architectural limits of ResNet-50 compared to more modern paradigms on the HAM10000 benchmark.\u003c/p\u003e","description":"","filename":"ResNet50ROCCurve.png","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/3053f2864d959dc2da1e6198.png"},{"id":106416560,"identity":"43ce474d-b1bb-43d6-85c3-7d80dc74292a","added_by":"auto","created_at":"2026-04-08 10:46:22","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":7410500,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/d74c3617-da8a-4c09-bd90-431655050177.pdf"},{"id":106381495,"identity":"c46bfe33-429a-4d52-b1fe-70d9b40aacb6","added_by":"auto","created_at":"2026-04-08 05:22:30","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":9900,"visible":true,"origin":"","legend":"","description":"","filename":"ModelsResults.docx","url":"https://assets-eu.researchsquare.com/files/rs-9285260/v1/a4ee60c12d3000b50822b76e.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eOptimizing Deep Learning for Skin Cancer: A Comparative Study of Convolutional and Attention-Based Models\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"1. Introduction","content":"\u003cp\u003eSkin cancers are the most commonly diagnosed group of cancers worldwide, with more than 1.5\u0026nbsp;million new cases estimated in 2022 \u003csup\u003e[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]\u003c/sup\u003e. Among these, melanoma represents a disproportionate share of mortality; if current incidence rates continue, the global burden of melanoma is projected to reach 510,000 new cases and 96,000 deaths annually by 2040, representing a roughly 50% increase in incidence from 2020 figures \u003csup\u003e[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/sup\u003e. The clinical consequences of delayed diagnosis are severe, as early-stage intervention substantially improves prognosis and reduces the need for aggressive surgical treatment \u003csup\u003e[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eCurrent standard-of-care diagnosis relies on visual inspection augmented by dermoscopy. However, this process is both time-intensive and subject to inter-observer variability. Even with the combination of visual examination and dermoscopic imaging, the absolute diagnostic accuracy for melanoma detection among experienced dermatologists ranges from 75 to 84% \u003csup\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/sup\u003e. This ceiling reflects not only the inherent difficulty of distinguishing morphologically similar lesion subtypes, but also systemic constraints including limited specialist availability, particularly in underserved and rural regions \u003csup\u003e[\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eThe application of deep convolutional neural networks to dermoscopic image analysis has demonstrated substantial promise in bridging this diagnostic gap. Transfer learning from large-scale image classification benchmarks such as ImageNet has emerged as the dominant paradigm, allowing models pretrained on millions of natural images to be adapted to medical imaging tasks with comparatively small domain-specific datasets \u003csup\u003e[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]\u003c/sup\u003e. Tschandl et al. introduced the HAM10000 dataset as a large-scale, multi-source collection of dermoscopic images spanning seven clinically relevant lesion categories, providing a standardized benchmark for evaluating such approaches \u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003ePrior work in this domain has focused predominantly on CNN-based architectures. Lin and Ghanta \u003csup\u003e[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]\u003c/sup\u003e compared MobileNetV2, ResNet50V2, EfficientNetV2B0, and VGG16 on the ISIC dataset, finding VGG16 to achieve the highest test accuracy of 84.7% and an AUC of 0.95 in a binary classification setting. However, the emergence of attention-based Vision Transformers \u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/sup\u003e and hybrid architectures such as the Swin Transformer \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e has introduced a fundamentally different inductive bias \u0026mdash; one that models long-range spatial dependencies through self-attention rather than local convolutional filters. Whether these architectural differences translate into measurable performance gains in the specific context of dermoscopic classification, and how such models interact in ensemble settings, remains an open question.\u003c/p\u003e \u003cp\u003eThis study addresses that question directly. We make the following contributions: (1) a rigorous, patient-level comparison of five architectures spanning CNN and Transformer families on the 7-class HAM10000 benchmark; (2) an analysis of architecture-specific training considerations, including the impact of learning rate scheduling on Transformer fine-tuning stability; (3) an ensemble strategy combining a CNN-based and a Transformer-based model, optimized via validation-set weight search; and (4) explainability visualizations through Grad-CAM and attention maps that illuminate the spatial reasoning of each model family.\u003c/p\u003e"},{"header":"2. Related Work","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003ch2\u003e2.1 CNN-Based Approaches on HAM10000\u003c/h2\u003e \u003cp\u003eThe HAM10000 dataset, introduced by Tschandl et al. \u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/sup\u003e, rapidly became the dominant benchmark for multi-class dermoscopic lesion classification. Early transfer learning studies demonstrated that pretrained CNN backbones could be effectively adapted to this domain. Rezvantalab et al. applied DenseNet201, ResNet152, InceptionV3, and InceptionResNetV2 to the HAM10000 benchmark, with DenseNet201 achieving an accuracy of 86.59% \u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]\u003c/sup\u003e. Garg et al. combined VGG16 and ResNet50 with ensemble metalearners including random forest, XGBoost, and SVM, reporting an accuracy of 90.51% on the same seven-class task \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e. Lin and Ghanta \u003csup\u003e[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]\u003c/sup\u003e further benchmarked four CNN architectures in a binary classification setting, with VGG16 achieving the highest AUC of 0.95. These studies collectively established the feasibility of transfer learning for dermoscopic classification, yet remained confined to the CNN paradigm and, in several cases, applied random rather than patient-level data splitting, potentially inflating reported metrics through lesion leakage.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec4\" class=\"Section2\"\u003e \u003ch2\u003e2.2 Transformer-Based Approaches\u003c/h2\u003e \u003cp\u003eThe introduction of the Vision Transformer (ViT) by Dosovitskiy et al. \u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/sup\u003e, which models image patches as sequential tokens processed through multi-head self-attention, opened a new research direction for medical image analysis. A systematic review of transformer applications in skin lesion classification identified 57 relevant studies published between 2017 and 2023, noting that HAM10000 was the most widely used benchmark across this body of work \u003csup\u003e[\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]\u003c/sup\u003e. Ayas \u003csup\u003e[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]\u003c/sup\u003e proposed a dedicated Swin Transformer model for multi-class skin lesion classification, demonstrating that the hierarchical shifted-window attention mechanism translates effectively to dermoscopic feature extraction. Subsequent work proposed SkinSwinViT, integrating Swin Transformer with a global attention mechanism, and demonstrated superior performance compared to prior CNN-based approaches \u003csup\u003e[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]\u003c/sup\u003e. Despite this progress, direct comparative evaluation of CNN and Transformer families under controlled, patient-level experimental conditions on the full seven-class HAM10000 task remains limited.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec5\" class=\"Section2\"\u003e \u003ch2\u003e2.3 Ensemble Methods\u003c/h2\u003e \u003cp\u003eEnsemble learning has consistently yielded performance gains in skin lesion classification by exploiting the complementary feature representations of constituent models. Liu et al. developed SkinNet using stacking ensemble methods, reporting an accuracy of 86.7% and an AUC of 0.96 on HAM10000 \u003csup\u003e[\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]\u003c/sup\u003e. A more recent study introduced an ensemble framework integrating Swin Transformer, ViT, and EfficientNet-B4, achieving 98.5% accuracy on a custom clinical dataset, with the authors explicitly attributing the performance gain to the complementary inductive biases of CNN and Transformer architectures \u003csup\u003e[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]\u003c/sup\u003e. EF-SwinNet, a hybrid model combining EfficientNet and Swin Transformer, further demonstrated that CNN-Transformer fusion addresses the limitation of CNNs in capturing long-range spatial dependencies while retaining parameter efficiency \u003csup\u003e[\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]\u003c/sup\u003e. The present study builds directly on this insight, evaluating a post-hoc weighted ensemble and searching for optimal combination weights on the validation set.\u003c/p\u003e \u003c/div\u003e"},{"header":"3. Methodology","content":"\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e \u003ch2\u003e3.1 Dataset and Patient-Level Splitting\u003c/h2\u003e \u003cp\u003eAll experiments in this study use the HAM10000 dataset \u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/sup\u003e, comprising 10,015 dermoscopic images across seven lesion classes: Melanocytic Nevi (NV), Melanoma (MEL), Benign Keratosis (BKL), Basal Cell Carcinoma (BCC), Actinic Keratoses (AKIEC), Vascular Lesions (VASC), and Dermatofibroma (DF). The dataset exhibits pronounced class imbalance, with Melanocytic Nevi accounting for nearly 67% of all samples.\u003c/p\u003e \u003cp\u003eA critical methodological consideration in skin lesion datasets is the presence of multiple images corresponding to the same lesion, which can introduce data leakage if splitting is performed at the image level. To prevent this, we applied patient-level partitioning using GroupShuffleSplit on the lesion_id field, ensuring that all images of a given lesion appear exclusively within one split. The resulting distribution allocates 72% of images to training, 13% to validation, and 15% to testing. We verified the absence of lesion-level overlap across all three splits via set intersection checks.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003e3.2 Class Imbalance Handling\u003c/h2\u003e \u003cp\u003eTo address the severe class imbalance, we employed WeightedRandomSampler during training, assigning each training sample a weight inversely proportional to its class frequency. This ensures that minority classes such as Dermatofibroma and Vascular Lesions are oversampled to appear with frequency comparable to the dominant Melanocytic Nevi class during each training epoch, without modifying the underlying dataset distribution.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec9\" class=\"Section2\"\u003e \u003ch2\u003e3.3 Image Preprocessing and Augmentation\u003c/h2\u003e \u003cp\u003eAll images were resized to the native input resolution of each architecture. CNN-based models and Swin Transformer-Base were trained at 224\u0026times;224 pixels, while EfficientNet-B4 was trained at its recommended resolution of 380\u0026times;380 pixels. Pixel values were normalized using ImageNet mean and standard deviation across all models. Training-time augmentation was restricted to geometric transformations \u0026mdash; horizontal flip, vertical flip, 90-degree rotation, and moderate shift-scale-rotate \u0026mdash; to avoid introducing artificial color or texture variations that might confound lesion appearance.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec10\" class=\"Section2\"\u003e \u003ch2\u003e3.4 Model Architectures\u003c/h2\u003e \u003cp\u003eWe evaluate five pretrained architectures spanning two architectural paradigms. ResNet-50 \u003csup\u003e[11r]\u003c/sup\u003e serves as the baseline CNN, comprising 50 layers with residual connections that address the vanishing gradient problem (~\u0026thinsp;25M parameters). EfficientNet-B4 \u003csup\u003e[12r]\u003c/sup\u003e applies compound scaling across depth, width, and resolution (~\u0026thinsp;19M parameters, trained at 380\u0026times;380). ConvNeXt-Base \u003csup\u003e[13r]\u003c/sup\u003e modernizes the CNN design by incorporating depthwise convolutions and LayerNorm while retaining convolutional inductive bias (~\u0026thinsp;89M parameters). Swin Transformer-Base \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e partitions input images into non-overlapping patches and processes them through shifted window attention, enabling hierarchical feature extraction (~\u0026thinsp;88M parameters). ViT-B/16 \u003csup\u003e[\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]\u003c/sup\u003e applies pure multi-head self-attention across a sequence of 196 non-overlapping 16\u0026times;16 patches (~\u0026thinsp;86M parameters).\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003e3.5 Training Configuration\u003c/h2\u003e \u003cp\u003eAll models were trained using AdamW optimization \u003csup\u003e[\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]\u003c/sup\u003e with a weight decay of 1\u0026times;10⁻\u0026sup2;. The loss function was cross-entropy with label smoothing of 0.1 \u003csup\u003e[\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]\u003c/sup\u003e, which penalizes overconfident predictions and improves calibration. Mixed-precision training (fp16) was applied to reduce memory consumption and training time. Gradient clipping with a maximum norm of 1.0 was applied at each update step.\u003c/p\u003e \u003cp\u003eCNN-based models were trained with CosineAnnealingWarmRestarts scheduling \u003csup\u003e[\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]\u003c/sup\u003e with T₀=10 and T_mult\u0026thinsp;=\u0026thinsp;2. Transformer-based models required a modified scheduling strategy: a linear warmup phase gradually increases the learning rate from zero to its target value before transitioning to cosine decay without restarts. This modification is motivated by the sensitivity of attention weight initialization to large gradient updates in the early training phase, as discussed in the original Swin Transformer paper \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e. Swin Transformer-Base additionally uses stochastic depth regularization (drop_path_rate\u0026thinsp;=\u0026thinsp;0.3) \u003csup\u003e[\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]\u003c/sup\u003e, and ViT-B/16 uses both stochastic depth (drop_path_rate\u0026thinsp;=\u0026thinsp;0.1) and dropout (rate\u0026thinsp;=\u0026thinsp;0.1). Early stopping was applied with a patience of 7\u0026ndash;8 epochs.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec12\" class=\"Section2\"\u003e \u003ch2\u003e3.6 Ensemble Strategy\u003c/h2\u003e \u003cp\u003eThe ensemble combines the softmax probability outputs of EfficientNet-B4 and Swin Transformer-Base through weighted averaging. The combination weight was optimized via a coarse grid search over the range [0.0, 1.0] in steps of 0.1, evaluated on the validation set. The resulting optimal weights of w\u0026thinsp;=\u0026thinsp;0.4 for EfficientNet-B4 and w\u0026thinsp;=\u0026thinsp;0.6 for Swin Transformer-Base were applied to the test set for final evaluation.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec13\" class=\"Section2\"\u003e \u003ch2\u003e3.7 Evaluation Metrics\u003c/h2\u003e \u003cp\u003eModel performance is reported across four metrics: test accuracy, balanced accuracy (mean per-class recall), macro-averaged F1-score, and macro-averaged AUC-ROC. The latter three metrics provide class-imbalance-robust evaluation, equally weighting all seven lesion categories regardless of their prevalence in the test set.\u003c/p\u003e \u003c/div\u003e"},{"header":"4. Results","content":"\u003cdiv id=\"Sec15\" class=\"Section2\"\u003e \u003ch2\u003e4.1 Individual Model Performance\u003c/h2\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e summarizes the test set performance of all five models across the four evaluation metrics. ResNet-50, serving as the CNN baseline, achieved a test accuracy of 75.90% and a macro F1-score of 0.6681, establishing a lower reference bound. EfficientNet-B4 improved substantially over this baseline, reaching 85.07% test accuracy and a macro AUC-ROC of 0.9712. ConvNeXt-Base achieved a comparable test accuracy of 84.74%, with notably stronger balanced accuracy (76.34%) and macro F1-score (0.7510), suggesting superior handling of the minority classes.\u003c/p\u003e \u003cp\u003eAmong Transformer-based architectures, Swin Transformer-Base achieved the highest balanced accuracy (79.66%) and AUC-ROC (0.9717) of any individual model. ViT-B/16 attained the highest single-model test accuracy at 85.66%, with a macro F1-score of 0.7741.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eTest set performance of all evaluated models on HAM10000.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003e Model\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTest Acc.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBalanced Acc.\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMacro F1\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAUC-ROC\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eResNet-50 (Baseline)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.7590\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.7500\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.6681\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.9548\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eEfficientNet-B4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.8507\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.7083\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.7380\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.9712\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eConvNeXt-Base\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.8474\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.7634\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.7510\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.9428\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSwin Transformer-Base\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.8507\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.7966\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.7719\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.9717\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eViT-B/16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e0.8566\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e0.7720\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.7741\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e0.9629\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e\u003cb\u003eEnsemble (EfficientNet-B4\u0026thinsp;+\u0026thinsp;Swin)\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e\u003cb\u003e0.8657\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e\u003cb\u003e0.7998\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e\u003cb\u003e0.7856\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e\u003cb\u003e0.9811\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec16\" class=\"Section2\"\u003e \u003ch2\u003e4.2 Training Dynamics\u003c/h2\u003e \u003cp\u003eA consistent pattern of overfitting was observed across all models, with training accuracy converging to values between 97% and 99% while validation accuracy plateaued at substantially lower levels, reflecting a generalization gap of approximately 13\u0026ndash;14 percentage points. This gap is consistent with the limited scale of HAM10000 relative to the capacity of the evaluated architectures, and aligns with findings reported in prior work on the same benchmark \u003csup\u003e[\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003cp\u003eTransformer-based models exhibited markedly different training dynamics compared to CNNs. When trained with CosineAnnealingWarmRestarts scheduling, Swin Transformer-Base displayed severe validation accuracy oscillations at each learning rate restart. This instability was eliminated upon replacing the restart-based scheduler with linear warmup followed by smooth cosine decay, as prescribed in the original Swin Transformer publication \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e. ViT-B/16 required an extended warmup period of five epochs and a lower peak learning rate of 3\u0026times;10⁻⁵ to achieve stable convergence.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec17\" class=\"Section2\"\u003e \u003ch2\u003e4.3 Ensemble Optimization and Final Performance\u003c/h2\u003e \u003cp\u003eThe ensemble weight search on the validation set identified w\u0026thinsp;=\u0026thinsp;0.4 for EfficientNet-B4 and w\u0026thinsp;=\u0026thinsp;0.6 for Swin Transformer-Base as optimal. The ensemble achieved a test accuracy of 86.57%, a balanced accuracy of 79.98%, a macro F1-score of 0.7856, and a macro AUC-ROC of 0.9811, outperforming all individual models across every reported metric.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec18\" class=\"Section2\"\u003e \u003ch2\u003e4.4 Explainability\u003c/h2\u003e \u003cp\u003eGrad-CAM visualizations for CNN-based models confirmed that activation patterns were spatially concentrated on lesion boundaries and internal texture regions. Attention maps generated for Swin Transformer-Base revealed broader spatial coverage distributed across the full lesion area. ViT-B/16 attention maps showed similar global coverage, attributable to the unrestricted receptive field of pure self-attention from the first layer. These qualitative differences in spatial reasoning are consistent with the quantitative finding that Transformer-based models achieve stronger balanced accuracy.\u003c/p\u003e \u003c/div\u003e"},{"header":"5. Discussion","content":"\u003cdiv id=\"Sec20\" class=\"Section2\"\u003e \u003ch2\u003e5.1 CNN vs. Transformer Performance\u003c/h2\u003e \u003cp\u003eThe results present a nuanced picture of the relative merits of convolutional and attention-based architectures. On overall test accuracy, ViT-B/16 narrowly outperforms all other individual models. However, on balanced accuracy and AUC-ROC \u0026mdash; metrics that weight minority class performance more equitably \u0026mdash; Swin Transformer-Base is the strongest individual model, achieving 79.66% balanced accuracy compared to 77.20% for ViT-B/16 and 70.83% for EfficientNet-B4.\u003c/p\u003e \u003cp\u003eThis divergence between accuracy and balanced accuracy is clinically significant. In a seven-class dermoscopic classification task where minority classes include malignant conditions such as Basal Cell Carcinoma and Actinic Keratoses, a model that achieves high overall accuracy through strong performance on the dominant Melanocytic Nevi class while underperforming on rarer malignancies is of limited clinical utility. The consistently stronger balanced accuracy of Transformer-based architectures suggests that attention-based models may be inherently better suited to imbalanced medical classification tasks by virtue of their capacity to model global image context without the locality bias imposed by convolutional kernels.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec21\" class=\"Section2\"\u003e \u003ch2\u003e5.2 The Role of Scheduling in Transformer Fine-Tuning\u003c/h2\u003e \u003cp\u003eA central finding of this study is that the choice of learning rate scheduler is not a secondary implementation detail but a primary determinant of whether Transformer fine-tuning converges stably. The CosineAnnealingWarmRestarts schedule, which is effective for CNN-based models, caused repeated destabilization of Swin Transformer training due to the hard learning rate resets that it imposes every T₀ epochs. Replacing the restart schedule with a smooth linear warmup followed by monotone cosine decay, as recommended in Liu et al. \u003csup\u003e[\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e]\u003c/sup\u003e, eliminated this instability entirely. This finding has practical implications for future work adapting Transformer architectures to small-scale medical datasets, and underscores the importance of architecture-specific training protocols rather than uniform hyperparameter application across model families.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec22\" class=\"Section2\"\u003e \u003ch2\u003e5.3 Ensemble Gains and Architectural Complementarity\u003c/h2\u003e \u003cp\u003eThe ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the strongest performance across all four reported metrics simultaneously, with an AUC-ROC of 0.9811 representing a gain of 0.0094 over the best individual model. These gains are consistent and appear across all metrics, suggesting that the combination captures genuine complementarity between the two model families. EfficientNet-B4 activations are concentrated on local texture and edge features, while Swin Transformer-Base attention spans the full lesion region. When combined through probability averaging, the ensemble benefits from both precise local feature discrimination and holistic spatial context modeling, consistent with findings reported in related ensemble studies \u003csup\u003e[\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]\u003c/sup\u003e.\u003c/p\u003e \u003c/div\u003e \u003cdiv id=\"Sec23\" class=\"Section2\"\u003e \u003ch2\u003e5.4 Limitations\u003c/h2\u003e \u003cp\u003eSeveral limitations merit acknowledgment. First, all experiments are conducted on a single dataset, and generalization to clinical deployment would require external validation on independent patient cohorts. Second, the limited scale of HAM10000 constrains the full expression of large Transformer capacity and is likely the primary driver of the generalization gap observed across all models. Third, the ensemble strategy is restricted to simple weighted averaging; more sophisticated combination methods may yield additional gains. Finally, the explainability visualizations provided are qualitative rather than quantitative, and rigorous evaluation against radiologist-annotated saliency maps would be necessary to draw definitive conclusions regarding clinical interpretability.\u003c/p\u003e \u003c/div\u003e"},{"header":"6. Conclusion","content":"\u003cp\u003eThis study presented a systematic evaluation of five deep learning architectures for the automated classification of seven skin lesion types using the HAM10000 dermoscopic benchmark. By comparing CNN-based models \u0026mdash; ResNet-50, EfficientNet-B4, and ConvNeXt-Base \u0026mdash; against attention-based Transformer architectures \u0026mdash; Swin Transformer-Base and ViT-B/16 \u0026mdash; under controlled, patient-level experimental conditions, this work provides a principled assessment of the relative strengths and limitations of each architectural family for this clinically important task.\u003c/p\u003e \u003cp\u003eThe results demonstrate that Transformer-based architectures achieve competitive or superior performance compared to state-of-the-art CNNs, with Swin Transformer-Base attaining the strongest balanced accuracy and AUC-ROC among individual models, and ViT-B/16 achieving the highest overall test accuracy. A key practical finding is that architecture-specific training protocols are essential for Transformer fine-tuning: the linear warmup followed by smooth cosine decay is necessary for stable convergence, in contrast to the restart-based scheduling effective for CNNs.\u003c/p\u003e \u003cp\u003eThe ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the strongest performance across all four evaluated metrics simultaneously \u0026mdash; test accuracy of 86.57%, balanced accuracy of 79.98%, macro F1-score of 0.7856, and macro AUC-ROC of 0.9811 \u0026mdash; consistent with the architectural complementarity between local convolutional feature extraction and global attention-based spatial reasoning.\u003c/p\u003e \u003cp\u003eFuture work should prioritize external validation on independent clinical cohorts, investigation of more sophisticated ensemble combination strategies, and exploration of data augmentation techniques specifically designed to address the class imbalance and limited scale of dermoscopic datasets. The integration of patient metadata alongside image features represents an additional avenue with demonstrated potential for performance improvement in this domain \u003csup\u003e[19m]\u003c/sup\u003e.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eFerlay J, Ervik M, Lam F et al (2024) Global Cancer Observatory: Cancer Today. Lyon: International Agency for Research on Cancer; Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://gco.iarc.who.int/today\u003c/span\u003e\u003cspan address=\"https://gco.iarc.who.int/today\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSaleh GM, Litvinova M, Lakhiani DD et al (2020) Machine learning and deep learning methods for skin lesion classification and diagnosis: A systematic review. Diagnostics 11:1390\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eArnold M, Singh D, Laversanne M et al (2022) Global burden of cutaneous melanoma in 2020 and projections to 2040. JAMA Dermatol 158(5):495\u0026ndash;503\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrinker TJ, Hekler A, Utikal JS et al (2018) Skin cancer classification using convolutional neural networks: systematic review. J Med Internet Res 20(10):e11936\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKim HE, Cosa-Linan A, Santhanam N et al (2022) Transfer learning for medical image classification: a literature review. BMC Med Imaging 22(1):69\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTschandl P, Rosendahl C, Kittler H (2018) The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci Data 5:180161\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLin R, Ghanta S (2024) Transfer learning with convolutional neural network-based models for skin cancer classification. J Emerg Investig. ;7\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDosovitskiy A, Beyer L, Kolesnikov A et al (2021) An image is worth 16x16 words: Transformers for image recognition at scale. ICLR\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu Z, Lin Y, Cao Y et al (2021) Swin Transformer: Hierarchical vision transformer using shifted windows. ICCV\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHimel GMS, Islam MM, Al-Aff KA et al (2024) Skin cancer segmentation and classification using Vision Transformer for automatic analysis in dermatoscopy-based noninvasive digital system. Int J Biomed Imaging\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMishra NK, Bhatt C, Biswas M et al (2024) Transformers in skin lesion classification and diagnosis: a systematic review. medRxiv\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChen X, Li C, Zhang Y et al (2024) SkinSwinViT: a lightweight transformer-based method for multiclass skin lesion classification with enhanced generalization capabilities. Appl Sci 14(10):4005\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLiu X, Yu Z, Tan L et al (2024) Enhancing skin lesion diagnosis with ensemble learning. Proceedings of the 4th International Conference on Computer Science and Blockchain (CCSB)\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoshchilov I, Hutter F (2019) Decoupled weight decay regularization. ICLR\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSzegedy C, Vanhoucke V, Ioffe S et al (2016) Rethinking the inception architecture for computer vision. CVPR\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLoshchilov I, Hutter F (2017) SGDR: Stochastic gradient descent with warm restarts. ICLR\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHuang G, Sun Y, Liu Z et al (2016) Deep networks with stochastic depth. ECCV\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRasheed J, Jan B, Taher F et al (2025) Deep ensemble learning for multiclass skin lesion classification. Bioengineering 12(9):934\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAshraf YM et al (2024) EF-SwinNet: a hybrid EfficientNet-Swin Transformer model for skin lesion classification on HAM10000. ICRPSET\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Multi-Class Skin Lesion Classification, HAM10000 Benchmark, Swin Transformer-Base, Heterogeneous Ensemble Learning, Patient-Level Data Partitioning, GroupShuffleSplit, Class Imbalance Mitigation, Stochastic Depth Regularization","lastPublishedDoi":"10.21203/rs.3.rs-9285260/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9285260/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eSkin cancer is among the most prevalent malignancies worldwide, with over 1.5\u0026nbsp;million new cases estimated in 2022 alone according to GLOBOCAN data \u003csup\u003e[\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]\u003c/sup\u003e. Despite the availability of dermoscopy, experienced dermatologists achieve a melanoma detection sensitivity of approximately 75\u0026ndash;84% using visual examination, a rate that underscores the diagnostic limitations of unaided clinical assessment \u003csup\u003e[\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]\u003c/sup\u003e. This study presents a systematic comparison of five deep learning architectures for the automated classification of seven skin lesion types using the HAM10000 dataset \u003csup\u003e[\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]\u003c/sup\u003e, comprising 10,015 dermoscopic images. We evaluate four architectures spanning both convolutional and attention-based paradigms: ResNet-50, EfficientNet-B4, ConvNeXt-Base, Swin Transformer-Base, and Vision Transformer (ViT-B/16). To address the pronounced class imbalance inherent in the dataset, we employed patient-level data partitioning via GroupShuffleSplit to prevent lesion leakage across splits, and WeightedRandomSampler during training. All models were trained using AdamW optimization with label smoothing and mixed-precision training. Transformer-based architectures were further stabilized through linear warmup scheduling and stochastic depth regularization. Our best single model, ViT-B/16, achieved a test accuracy of 85.66% and a macro AUC-ROC of 0.9629. An ensemble of EfficientNet-B4 and Swin Transformer-Base achieved the highest overall performance with a test accuracy of 86.57%, a balanced accuracy of 79.98%, a macro F1-score of 0.7856, and a macro AUC-ROC of 0.9811. These results demonstrate that heterogeneous ensemble strategies combining architecturally diverse models offer a meaningful improvement over individual classifiers in dermoscopic lesion classification.\u003c/p\u003e","manuscriptTitle":"Optimizing Deep Learning for Skin Cancer: A Comparative Study of Convolutional and Attention-Based Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-08 05:22:25","doi":"10.21203/rs.3.rs-9285260/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a5552d4e-530f-4877-b89f-5c693258b2bd","owner":[],"postedDate":"April 8th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":65501109,"name":"Artificial Intelligence and Machine Learning"}],"tags":[],"updatedAt":"2026-04-08T05:22:25+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-08 05:22:25","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9285260","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9285260","identity":"rs-9285260","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}