A Novel Multi-Stage Fusion Pipeline for Robust and Interpretable Melanoma Classification Using Physics-Informed and Vision-Language Models

A Novel Multi-Stage Fusion Pipeline for Robust and Interpretable Melanoma Classification Using Physics-Informed and Vision-Language Models | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Novel Multi-Stage Fusion Pipeline for Robust and Interpretable Melanoma Classification Using Physics-Informed and Vision-Language Models G. Isha, F. D Asbel sherlin This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8785131/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 16 You are reading this latest preprint version Abstract Melanoma, a highly aggressive form of skin cancer, requires early and accurate diagnosis to improve patient survival, yet existing deep learning methods often struggle with noise, limited labeled data, poor calibration, and cross-domain generalization. This paper proposes a novel, modular AI-driven pipeline for melanoma classification from dermoscopic images that integrates physics-informed preprocessing, diffusion–transformer-based segmentation, hybrid self-supervised representation learning, privacy-preserving federated classification, and uncertainty-aware explainability. Physics-Informed AI-Based Denoising Preprocessing (PIAIDP) effectively suppresses hair, glare, and illumination artifacts while preserving lesion boundaries, achieving 32.4 dB PSNR, 0.93 SSIM, and the lowest Hair IoU Loss of 0.094. Diffusion–Transformer Segmentation Fusion (DTSF) combines denoising diffusion models with TransUNet via multi-scale cross-attention, attaining a Dice score of 0.902, Jaccard index of 0.849, and reduced boundary error (HD95 = 7.6). From the segmented lesions, Contrastive Self-Supervised Hybrid Graph–ViT Embedding (CSS-HGVE) fuses structural graph representations with transformer-based visual features, yielding the highest linear probe accuracy of 90.8% and improved class separability (silhouette score = 0.51). For classification, the Prompt-Guided Multi-Task Federated Classifier (PG-MTFC) leverages domain-guided CLIP prompts and Evidential Deep Learning to jointly predict melanoma classes and quantify uncertainty. Under non-IID federated settings with differential privacy (ε ≤ 2.0), the proposed model achieves 92.6% accuracy, F1-score of 0.908, and AUROC of 0.949, with superior calibration (ECE = 0.050). The Uncertainty-Aware Multi-Modal Explainability (UAMME) module further enhances interpretability, achieving higher faithfulness (0.768) and clinician trust (4.4/5). Extensive evaluation on ISIC 2018, PH2, and Derm7pt datasets demonstrates improved robustness, fairness, and cross-dataset generalization, supporting the framework’s applicability for trustworthy and privacy-preserving clinical melanoma analysis. Melanoma classification Dermoscopic image analysis Physics-informed deep learning Diffusion-transformer segmentation Federated learning Multi-modal explainability Uncertainty estimation Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 25 Apr, 2026 Reviews received at journal 24 Apr, 2026 Reviews received at journal 22 Apr, 2026 Reviews received at journal 21 Apr, 2026 Reviews received at journal 17 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviews received at journal 15 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers agreed at journal 15 Apr, 2026 Reviewers invited by journal 25 Feb, 2026 Editor assigned by journal 11 Feb, 2026 Submission checks completed at journal 10 Feb, 2026 First submitted to journal 04 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8785131","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":597130023,"identity":"1aa04e75-2791-424c-abf7-b231590cdcd4","order_by":0,"name":"G. Isha","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABPUlEQVRIie2RMUvDQBTHX3iQOFy6uFyImK9woRAtVDv6NSyBZElRcBEULATOJda1Tn4FXTorhfQrFK5DpZDJIaVQghTxWhykScBRML/h8d7jfvy5O4CKir8Igvqjh0NQtfB1mspJ1X6nUKiR2LX7awVLc7aUfRo4u+R7LMI60ZJpBsdnTIRvi/NraqnU87D5MbBqCEo6D3KKPSQHdgRu42kS181+TG1OZvGs0xM2R0DjYZBXQqJSAsjY+NRB2Stc8/x6JxKKVFTUixQtMVZwIxV/ieSTtjgEjtmIRKtMsRAck8BQKoGDOqdtviMVyES7TGFIHHOPjZgxDi5MvUddvn7ku65wOSph0V2s+1FivF9esdrYf16QZfPo8VZ+ZbYSmyadF6S8bOr2WuGb2s2dX6cUbgFWxeuKioqKf8kX++hiOCLEd9gAAAAASUVORK5CYII=","orcid":"","institution":"Anna University, Chennai","correspondingAuthor":true,"prefix":"","firstName":"G.","middleName":"","lastName":"Isha","suffix":""},{"id":597130024,"identity":"0b62cdc8-9cfe-48ae-ac38-2e3cc2374f43","order_by":1,"name":"F. D Asbel sherlin","email":"","orcid":"","institution":"Anna University, Chennai","correspondingAuthor":false,"prefix":"","firstName":"F.","middleName":"D Asbel","lastName":"sherlin","suffix":""}],"badges":[],"createdAt":"2026-02-04 10:38:57","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8785131/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8785131/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":104400086,"identity":"622e9ac4-df60-4caa-b8c5-0b484fe4f0db","added_by":"auto","created_at":"2026-03-11 12:08:49","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":999616,"visible":true,"origin":"","legend":"","description":"","filename":"IJCIS.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8785131/v1_covered_a6c396b7-fd63-46e6-9893-657d63458f46.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Novel Multi-Stage Fusion Pipeline for Robust and Interpretable Melanoma Classification Using Physics-Informed and Vision-Language Models","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-computational-intelligence-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [International Journal of Computational Intelligence Systems](https://link.springer.com/journal/44196)","snPcode":"44196","submissionUrl":"https://submission.springernature.com/new-submission/44196/3","title":"International Journal of Computational Intelligence Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Open","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Melanoma classification, Dermoscopic image analysis, Physics-informed deep learning, Diffusion-transformer segmentation, Federated learning, Multi-modal explainability, Uncertainty estimation","lastPublishedDoi":"10.21203/rs.3.rs-8785131/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8785131/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eMelanoma, a highly aggressive form of skin cancer, requires early and accurate diagnosis to improve patient survival, yet existing deep learning methods often struggle with noise, limited labeled data, poor calibration, and cross-domain generalization. This paper proposes a novel, modular AI-driven pipeline for melanoma classification from dermoscopic images that integrates physics-informed preprocessing, diffusion\u0026ndash;transformer-based segmentation, hybrid self-supervised representation learning, privacy-preserving federated classification, and uncertainty-aware explainability. Physics-Informed AI-Based Denoising Preprocessing (PIAIDP) effectively suppresses hair, glare, and illumination artifacts while preserving lesion boundaries, achieving 32.4 dB PSNR, 0.93 SSIM, and the lowest Hair IoU Loss of 0.094. Diffusion\u0026ndash;Transformer Segmentation Fusion (DTSF) combines denoising diffusion models with TransUNet via multi-scale cross-attention, attaining a Dice score of 0.902, Jaccard index of 0.849, and reduced boundary error (HD95\u0026thinsp;=\u0026thinsp;7.6). From the segmented lesions, Contrastive Self-Supervised Hybrid Graph\u0026ndash;ViT Embedding (CSS-HGVE) fuses structural graph representations with transformer-based visual features, yielding the highest linear probe accuracy of 90.8% and improved class separability (silhouette score\u0026thinsp;=\u0026thinsp;0.51). For classification, the Prompt-Guided Multi-Task Federated Classifier (PG-MTFC) leverages domain-guided CLIP prompts and Evidential Deep Learning to jointly predict melanoma classes and quantify uncertainty. Under non-IID federated settings with differential privacy (ε\u0026thinsp;\u0026le;\u0026thinsp;2.0), the proposed model achieves 92.6% accuracy, F1-score of 0.908, and AUROC of 0.949, with superior calibration (ECE\u0026thinsp;=\u0026thinsp;0.050). The Uncertainty-Aware Multi-Modal Explainability (UAMME) module further enhances interpretability, achieving higher faithfulness (0.768) and clinician trust (4.4/5). Extensive evaluation on ISIC 2018, PH2, and Derm7pt datasets demonstrates improved robustness, fairness, and cross-dataset generalization, supporting the framework\u0026rsquo;s applicability for trustworthy and privacy-preserving clinical melanoma analysis.\u003c/p\u003e","manuscriptTitle":"A Novel Multi-Stage Fusion Pipeline for Robust and Interpretable Melanoma Classification Using Physics-Informed and Vision-Language Models","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-03-02 18:19:21","doi":"10.21203/rs.3.rs-8785131/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-04-25T11:16:44+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-24T04:56:51+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-22T19:35:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-21T06:50:37+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-17T19:06:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"153168900641910849020657744499662834904","date":"2026-04-15T16:26:04+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-15T13:49:49+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"221997126186218979737914044880768353430","date":"2026-04-15T11:25:54+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"87688550064080019032348588887702641505","date":"2026-04-15T11:01:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"147737999501095412547944243914772546471","date":"2026-04-15T07:11:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"145249017568288580825051444768007201847","date":"2026-04-15T06:40:04+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"279399045274313849183685858925032033153","date":"2026-04-15T06:03:58+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-25T19:38:28+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-11T17:00:36+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-11T02:55:44+00:00","index":"","fulltext":""},{"type":"submitted","content":"International Journal of Computational Intelligence Systems","date":"2026-02-04T09:52:53+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"international-journal-of-computational-intelligence-systems","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"Learn more about [International Journal of Computational Intelligence Systems](https://link.springer.com/journal/44196)","snPcode":"44196","submissionUrl":"https://submission.springernature.com/new-submission/44196/3","title":"International Journal of Computational Intelligence Systems","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Springer Open","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0a3f3b66-3a55-4755-82a6-1852f1ff676d","owner":[],"postedDate":"March 2nd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-04-25T11:24:00+00:00","versionOfRecord":[],"versionCreatedAt":"2026-03-02 18:19:21","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8785131","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8785131","identity":"rs-8785131","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}