A Multimodal Clinical Decision Support System for Retinal Diseases Detection and Personalized Disease Progression and Severity Analysis

A Multimodal Clinical Decision Support System for Retinal Diseases Detection and Personalized Disease Progression and Severity Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article A Multimodal Clinical Decision Support System for Retinal Diseases Detection and Personalized Disease Progression and Severity Analysis Vedaant Agarwal, Yelena Yesha, Rose Yesha This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-9004546/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 15 You are reading this latest preprint version Abstract Objectives Clinical Decision Support Systems (CDSSs) have potential to enhance retinal diagnosis from optical coherence tomography (OCT) scans. However, current CDSSs face three critical limitations. Firstly, high-performing CDSSs require extensive preprocessing and fail with speckle, noise-like pattern formed from light scattering off retinal microstructures. Secondly, current CDSSs do not quantify disease severity and finally the lack clinical interpretability, preventing clinical adoption. Methods: A multimodal CDSS could be developed addressing each limitation. The Speckle-Aware Dynamic Vision Transformer (SA-DVT) framework leverages speckle as a diagnostic cue for feature extraction from OCT images. The Severity Estimation and Personalized Analysis (SEPIA) framework provided severity estimation through computing Euclidean distance from healthy anatomy. The Clinical Reasoning and Analysis Framework for Trust utilizes t-SNE dimensionality reduction, Swin-UNet segmentation, and GPT-5 patient report generation to create interpretable outputs across graphical, visual, and textual modalities. Results: SA-DVT achieved 96.9% disease classification accuracy with precision and recall exceeding 96% across disease categories. SEPIA achieved 96.6% bootstrap confidence for longitudinal tracking. Dimensionality reduction extracted SA-DVT and SEPIA feature vectors in 2D space for analyzing disease classification patterns. Retinal layer segmentation achieved 0.9393 Dice coefficient, revealing biomarkers and layer deformations. Patient reports with retinal status, biomarkers, and clinical summaries demonstrated 100% consistency across five independent runs. Conclusion: This CDSS has potential as a clinical tool for retinal disease management. Clinical Decision Support System Disease Progression Personalized Analysis Retinal Disease Swin Transformer t-distributed Stochastic Neighbor Embedding Unsupervised Learning Vision Transformer Visual Impairment Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 02 May, 2026 Reviews received at journal 10 Apr, 2026 Reviewers agreed at journal 01 Apr, 2026 Reviews received at journal 01 Apr, 2026 Reviewers agreed at journal 31 Mar, 2026 Reviews received at journal 31 Mar, 2026 Reviewers agreed at journal 31 Mar, 2026 Reviewers agreed at journal 30 Mar, 2026 Reviewers agreed at journal 29 Mar, 2026 Reviews received at journal 29 Mar, 2026 Reviewers agreed at journal 29 Mar, 2026 Reviewers invited by journal 29 Mar, 2026 Editor assigned by journal 29 Mar, 2026 Submission checks completed at journal 07 Mar, 2026 First submitted to journal 01 Mar, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-9004546","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":615186517,"identity":"6363c665-bbd6-404e-b106-1030d174b216","order_by":0,"name":"Vedaant Agarwal","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABN0lEQVRIiWNgGAWjYHACA2SOhRwDMxJXArcWuDYJY7gWHmK1JDbAmLi0GBxv3vjh544/DPz9i5995qmRSN/Oznvwwc89Nvb2DMwHb/Ng0XLmWLFk7xkDBokbz4xn8xyTyN3ZzJds2PMsLbGHgS3ZGosWyRk5BhK8bUCH3ThgzJzDJpG74TCPmTTDgcMJPAxABjYt898Y//wL1CJ/4/hn5px/EukGh3nMfzMc+G/Pw8D/DZsWfgmgUSBbDM73GDPntkkkALWYMTMcOMDYw8DDhlULT1qZtWybMY/hDZ5i5r99EoY7m3mMJXsOJCf2HGYztpyDqYWN/fDmm2/b5OTkzh/fzDjjm428Of8Zww8/DtjZs7c3P7zxBksoQwEPg0QCNAzhYszYlSI58QC6llEwCkbBKBgFEAAAlihdtqRAybUAAAAASUVORK5CYII=","orcid":"","institution":"Pine Crest","correspondingAuthor":true,"prefix":"","firstName":"Vedaant","middleName":"","lastName":"Agarwal","suffix":""},{"id":615186520,"identity":"188523f6-d411-4572-9bf8-dec13a33c1f5","order_by":1,"name":"Yelena Yesha","email":"","orcid":"","institution":"U of Miami","correspondingAuthor":false,"prefix":"","firstName":"Yelena","middleName":"","lastName":"Yesha","suffix":""},{"id":615186523,"identity":"73a4e1ed-0982-4fb3-a2d1-2ae346810840","order_by":2,"name":"Rose Yesha","email":"","orcid":"","institution":"Georgetown University","correspondingAuthor":false,"prefix":"","firstName":"Rose","middleName":"","lastName":"Yesha","suffix":""}],"badges":[],"createdAt":"2026-03-02 01:38:03","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-9004546/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-9004546/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":106093100,"identity":"af0900b3-0409-48e1-b5e9-e6a1dd41c08e","added_by":"auto","created_at":"2026-04-03 11:34:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1453293,"visible":true,"origin":"","legend":"","description":"","filename":"AMultimodalDeepLearningFrameworkforANovelClinicalDecisionSupportSystemforRetinalDiseasesJournalofBigDataFormttedpaper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-9004546/v1_covered_1301c434-9964-4a8d-bb59-b0b8ff1a90c0.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"A Multimodal Clinical Decision Support System for Retinal Diseases Detection and Personalized Disease Progression and Severity Analysis","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Clinical Decision Support System, Disease Progression, Personalized Analysis, Retinal Disease, Swin Transformer, t-distributed Stochastic Neighbor Embedding, Unsupervised Learning, Vision Transformer, Visual Impairment","lastPublishedDoi":"10.21203/rs.3.rs-9004546/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-9004546/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eObjectives\u003c/h2\u003e \u003cp\u003eClinical Decision Support Systems (CDSSs) have potential to enhance retinal diagnosis from optical coherence tomography (OCT) scans. However, current CDSSs face three critical limitations. Firstly, high-performing CDSSs require extensive preprocessing and fail with speckle, noise-like pattern formed from light scattering off retinal microstructures. Secondly, current CDSSs do not quantify disease severity and finally the lack clinical interpretability, preventing clinical adoption. Methods: A multimodal CDSS could be developed addressing each limitation. The Speckle-Aware Dynamic Vision Transformer (SA-DVT) framework leverages speckle as a diagnostic cue for feature extraction from OCT images. The Severity Estimation and Personalized Analysis (SEPIA) framework provided severity estimation through computing Euclidean distance from healthy anatomy. The Clinical Reasoning and Analysis Framework for Trust utilizes t-SNE dimensionality reduction, Swin-UNet segmentation, and GPT-5 patient report generation to create interpretable outputs across graphical, visual, and textual modalities. Results: SA-DVT achieved 96.9% disease classification accuracy with precision and recall exceeding 96% across disease categories. SEPIA achieved 96.6% bootstrap confidence for longitudinal tracking. Dimensionality reduction extracted SA-DVT and SEPIA feature vectors in 2D space for analyzing disease classification patterns. Retinal layer segmentation achieved 0.9393 Dice coefficient, revealing biomarkers and layer deformations. Patient reports with retinal status, biomarkers, and clinical summaries demonstrated 100% consistency across five independent runs. Conclusion: This CDSS has potential as a clinical tool for retinal disease management.\u003c/p\u003e","manuscriptTitle":"A Multimodal Clinical Decision Support System for Retinal Diseases Detection and Personalized Disease Progression and Severity Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-01 20:09:27","doi":"10.21203/rs.3.rs-9004546/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-05-02T22:35:18+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-10T15:08:14+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"138470000191490878030673054012727025557","date":"2026-04-01T17:12:37+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-01T07:31:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"211031810513651593980371964844953429385","date":"2026-04-01T00:09:21+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-31T10:08:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"329637566650676308987433023996581339557","date":"2026-03-31T09:41:34+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"314504968610180186781607947401796759097","date":"2026-03-30T06:24:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"178562473081211560797015718424754881390","date":"2026-03-30T03:05:21+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-30T02:05:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"192654847694079156059623505969499870395","date":"2026-03-30T01:54:03+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-03-30T00:07:46+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-03-29T23:46:11+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-03-07T11:03:08+00:00","index":"","fulltext":""},{"type":"submitted","content":"Journal of Big Data","date":"2026-03-02T01:21:32+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"journal-of-big-data","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bigd","sideBox":"Learn more about [Journal of Big Data](http://journalofbigdata.springeropen.com)","snPcode":"40537","submissionUrl":"https://submission.nature.com/new-submission/40537/3","title":"Journal of Big Data","twitterHandle":"@SpringerOpen","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7505203f-72ff-426b-95d5-4d73846ab1a1","owner":[],"postedDate":"April 1st, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Revision requested","date":"2026-05-02T22:35:18+00:00","index":"","fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-05-02T22:38:36+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-01 20:09:27","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-9004546","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-9004546","identity":"rs-9004546","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}