A Multimodal Attention-Based Multi-Instance Learning Framework for Fair and Interpretable Pediatric Teledermatology

A Multimodal Attention-Based Multi-Instance Learning Framework for Fair and Interpretable Pediatric Teledermatology | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Multimodal Attention-Based Multi-Instance Learning Framework for Fair and Interpretable Pediatric Teledermatology Khondakar Ashik Shahriar This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8880185/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Purpose : Pediatric skin diseases are prevalent yet frequently underdiagnosed in low-resource settings across Sub-Saharan Africa due to limited access to specialized dermatological care. This study examines whether a subject-level multimodal learning framework can improve diagnostic accuracy, interpretability, and fairness in pediatric teledermatology across diverse skin types. Methods : A subject-level multimodal multi-instance learning framework is developed in which each patient is represented as a bag of clinical images, with visual features integrated alongside demographic and clinical metadata. A gated attention mechanism is employed to aggregate heterogeneous image instances into interpretable subject-level representations, while multimodal fusion provides contextual information for diagnosis. The framework is evaluated using the PASSION pediatric dermatology dataset across four common skin conditions. Ablation studies and statistical analyses are conducted to assess the contributions of attention-based aggregation and multimodal fusion. Fairness is evaluated across Fitzpatrick skin types. Results : The proposed framework achieves an overall classification accuracy of 82.8\% and a macro F1-score of 0.81. Ablation results demonstrate that gated attention-based aggregation significantly outperforms naive pooling strategies, while multimodal fusion further enhances diagnostic robustness. Fairness analysis indicates stable performance across Fitzpatrick skin types. Conclusion : Subject-level multimodal learning provides a robust, interpretable, and equitable approach for AI-assisted pediatric teledermatology, demonstrating strong potential for improving diagnostic access and quality of care in low-resource clinical environments. Medical Informatics Dermatology Multi-instance learning Pediatric teledermatology Multimodal learning Explainable AI Full Text Additional Declarations The authors declare no competing interests. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8880185","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":591359762,"identity":"d0d7ba1a-2405-4488-b37a-ef90261c3639","order_by":0,"name":"Khondakar Ashik Shahriar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGklEQVRIiWNgGAWjYBACCQkgwdggYcAgwcDGwGDAwMAPEk4oIEWLZANIiwFBLQxQLUBgcABM4tYiObuB8ePPHRbG8rObnz2uKLDLNz6/OvHDAwMGeX6xA1i1SMscYJbmPSNhZnDnmLnhGYNky2033m6WADrMcObsBKxa5CQSGKQZ2yRsDCQSzCQbDJgNzG6c3QDSkmBwG6cW5p8/gVrkZ6R/A2qpNzCecXbzD3xapCUS2CR42yTMGG7kgGw5bGDA37sNry2SMxLbrIFajA1u5JQBtRw3kLjBu80iwUACp18kbiQfvvmzrc5w/oz0bZINf6oN+PvPbr75o8JGnl8auxZwpKCZAlYpgUM5VsB/gBTVo2AUjIJRMAIAAAU9WY6TsmTrAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0009-0008-2571-7485","institution":"Bangladesh University of Engineering and Technology","correspondingAuthor":true,"prefix":"","firstName":"Khondakar","middleName":"Ashik","lastName":"Shahriar","suffix":""}],"badges":[],"createdAt":"2026-02-14 13:15:21","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8880185/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8880185/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":102963700,"identity":"569dd92b-2c37-4d42-8c48-c523084e523f","added_by":"auto","created_at":"2026-02-19 04:20:08","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":6257199,"visible":true,"origin":"","legend":"","description":"","filename":"Miccaiwork.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8880185/v1_covered_763062b7-fbc4-4f9f-9cf0-cfb80c54c6a5.pdf"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eA Multimodal Attention-Based Multi-Instance Learning Framework for Fair and Interpretable Pediatric Teledermatology\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Bangladesh University of Engineering and Technology","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Dermatology, Multi-instance learning, Pediatric teledermatology, Multimodal learning, Explainable AI","lastPublishedDoi":"10.21203/rs.3.rs-8880185/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8880185/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003ePurpose\u003c/strong\u003e: Pediatric skin diseases are prevalent yet frequently underdiagnosed in low-resource settings across Sub-Saharan Africa due to limited access to specialized dermatological care. This study examines whether a subject-level multimodal learning framework can improve diagnostic accuracy, interpretability, and fairness in pediatric teledermatology across diverse skin types. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e: A subject-level multimodal multi-instance learning framework is developed in which each patient is represented as a bag of clinical images, with visual features integrated alongside demographic and clinical metadata. A gated attention mechanism is employed to aggregate heterogeneous image instances into interpretable subject-level representations, while multimodal fusion provides contextual information for diagnosis. The framework is evaluated using the PASSION pediatric dermatology dataset across four common skin conditions. Ablation studies and statistical analyses are conducted to assess the contributions of attention-based aggregation and multimodal fusion. Fairness is evaluated across Fitzpatrick skin types. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e: The proposed framework achieves an overall classification accuracy of 82.8\\% and a macro F1-score of 0.81. Ablation results demonstrate that gated attention-based aggregation significantly outperforms naive pooling strategies, while multimodal fusion further enhances diagnostic robustness. Fairness analysis indicates stable performance across Fitzpatrick skin types. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e: Subject-level multimodal learning provides a robust, interpretable, and equitable approach for AI-assisted pediatric teledermatology, demonstrating strong potential for improving diagnostic access and quality of care in low-resource clinical environments.\u003c/p\u003e","manuscriptTitle":"A Multimodal Attention-Based Multi-Instance Learning Framework for Fair and Interpretable Pediatric Teledermatology","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-18 03:12:49","doi":"10.21203/rs.3.rs-8880185/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"231ead43-749c-4702-af29-1230ac942ae6","owner":[],"postedDate":"February 18th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":62932665,"name":"Medical Informatics"}],"tags":[],"updatedAt":"2026-02-18T03:12:49+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-18 03:12:49","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8880185","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8880185","identity":"rs-8880185","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}