Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making

Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making Aniket Patil, Viraj Patil, Sangram Sankpal, Tanuja Patankar, Harsha Bhute This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6430452/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Access to quality health care continues to be a major challenge for remote or overlooked regions that do not have the necessary medical infrastructures and human resources. This research proposes integrating structured data-laboratory results, vitals, and demographics- with unstructured medical data-clinical notes, free-text diagnosis-finalized into Multimodal Decision Support Systems (MDSS)-that closes the healthcare gap by enhancing the diagnostic accuracy and treatment recommendations. Our innovative approach entails employing Random Forest Classifiers for structured data and BERT-based embeddings for unstructured data and fuses their predictive outputs through late fusion technique. Among various evaluated fusion methods, including simple average, weighted average, and stacked fusion, the stacked fusion approach resulted in achieving maximum diagnostic accuracy, i.e., 87 against individual models, thus taking diagnostic accuracy improvement into huge consideration as well as significant reductions in misdiagnosis, in addition to last but not least, personalized healthcare recommendation, especially to rural populations. The evaluation of this system used the MIMIC-IV dataset and showed improved performance in risk prediction and analysis of patient outcomes. The first generation of our smart health care assistant will feature video consultations and multi-lingual support, as well as real-time processing capabilities to allow access to high-quality health care for these populations. Future improvements in optimizing data imputation, enhancing interpretability, and ensuring HIPAA and GDPR compliance will make for secure and ethical data usage. This work will build ground work for AI-Personal Healthcare Solutions with a long-term goal of bridging the gap between rural and urban patient populations in access to care. Multimodal Decision Support System Healthcare Accessibility Structured and Unstructured Data Random Forest Classifier BERT Late Fusion Telehealth Personalized Medicine Healthcare Data Privacy Rural Healthcare Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6430452","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":441912875,"identity":"a0713ca9-acf2-446b-a805-20fe6405cc03","order_by":0,"name":"Aniket Patil","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABLUlEQVRIie3RsUrEMBgH8IRCXSJdI0rvFVI6KHgPk1C4Thahi5NEArqcuFZuuFfo5jnZEqjLPUAHB497gRYcKlQwKYpDW7xRMP8hyZAf3/clAJiY/MlA8X24yiqC3R0JVbsNeZ6cn/i7lLG+CAASVReM/3bfWQhB6gZEzlLwDBEcPuzJzWu9kpEDrKJ+6xP8kguWUBDjAvIME3z2OJ/53v1axgfcDhZHA2VKJiSigHFNiCJpiezD/WvJ0gz5Fu6LiSatIktNqGqMaPKhyNMIIYoEanyWapIRTDsCdRWAfFj1iaeIN5/h2CsYzznBXjfL7TpkibQDa2AUtwy3uJlOI1c+b7Ztezk51i/2vjpldzdCwmb0rTH9aVUvkIPuwyw0SsAQ0ft4FRMTE5P/k0+pC3ByQlrRQAAAAABJRU5ErkJggg==","orcid":"","institution":"Pimpri Chinchwad College Of Engineering Pune","correspondingAuthor":true,"prefix":"","firstName":"Aniket","middleName":"","lastName":"Patil","suffix":""},{"id":441912876,"identity":"f6aee090-7277-47a4-bdb5-7f9700241611","order_by":1,"name":"Viraj Patil","email":"","orcid":"","institution":"Pimpri Chinchwad College Of Engineering Pune","correspondingAuthor":false,"prefix":"","firstName":"Viraj","middleName":"","lastName":"Patil","suffix":""},{"id":441912877,"identity":"ed529de1-6986-4bd1-9c50-5668f9f69d12","order_by":2,"name":"Sangram Sankpal","email":"","orcid":"","institution":"Pimpri Chinchwad College Of Engineering Pune","correspondingAuthor":false,"prefix":"","firstName":"Sangram","middleName":"","lastName":"Sankpal","suffix":""},{"id":441912878,"identity":"def15a38-0dce-4f69-9d6e-8085cc07a909","order_by":3,"name":"Tanuja Patankar","email":"","orcid":"","institution":"Pimpri Chinchwad College Of Engineering Pune","correspondingAuthor":false,"prefix":"","firstName":"Tanuja","middleName":"","lastName":"Patankar","suffix":""},{"id":441912879,"identity":"1333d8ae-c21a-4d57-bb9d-6efcbf485ac3","order_by":4,"name":"Harsha Bhute","email":"","orcid":"","institution":"Pimpri Chinchwad College Of Engineering Pune","correspondingAuthor":false,"prefix":"","firstName":"Harsha","middleName":"","lastName":"Bhute","suffix":""}],"badges":[],"createdAt":"2025-04-11 18:08:14","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6430452/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6430452/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":80792355,"identity":"61d58802-f9ed-4a7d-b7c9-4393fcc20c9d","added_by":"auto","created_at":"2025-04-17 06:54:49","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":229749,"visible":true,"origin":"","legend":"","description":"","filename":"Researchpaper.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6430452/v1_covered_bf352a5a-09c9-4f88-a67b-342f7e67615d.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Multimodal Decision Support System, Healthcare Accessibility, Structured and Unstructured Data, Random Forest Classifier, BERT, Late Fusion, Telehealth, Personalized Medicine, Healthcare Data Privacy, Rural Healthcare","lastPublishedDoi":"10.21203/rs.3.rs-6430452/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6430452/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Access to quality health care continues to be a major challenge for remote or overlooked regions that do not have the necessary medical infrastructures and human resources. This research proposes integrating structured data-laboratory results, vitals, and demographics- with unstructured medical data-clinical notes, free-text diagnosis-finalized into Multimodal Decision Support Systems (MDSS)-that closes the healthcare gap by enhancing the diagnostic accuracy and treatment recommendations. Our innovative approach entails employing Random Forest Classifiers for structured data and BERT-based embeddings for unstructured data and fuses their predictive outputs through late fusion technique. Among various evaluated fusion methods, including simple average, weighted average, and stacked fusion, the stacked fusion approach resulted in achieving maximum diagnostic accuracy, i.e., 87 against individual models, thus taking diagnostic accuracy improvement into huge consideration as well as significant reductions in misdiagnosis, in addition to last but not least, personalized healthcare recommendation, especially to rural populations. The evaluation of this system used the MIMIC-IV dataset and showed improved performance in risk prediction and analysis of patient outcomes. The first generation of our smart health care assistant will feature video consultations and multi-lingual support, as well as real-time processing capabilities to allow access to high-quality health care for these populations. Future improvements in optimizing data imputation, enhancing interpretability, and ensuring HIPAA and GDPR compliance will make for secure and ethical data usage. This work will build ground work for AI-Personal Healthcare Solutions with a long-term goal of bridging the gap between rural and urban patient populations in access to care.","manuscriptTitle":"Multimodal Decision Support System for Improved Diagnosis and Healthcare Decision Making","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-14 03:43:55","doi":"10.21203/rs.3.rs-6430452/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1fae2b40-eb5a-4544-8b63-a3621853d9fb","owner":[],"postedDate":"April 14th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-04-17T06:54:10+00:00","versionOfRecord":[],"versionCreatedAt":"2025-04-14 03:43:55","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6430452","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6430452","identity":"rs-6430452","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}