Development and validation of random-forest based federated learning algorithms for delirium prediction using electronic medical records from eleven hospitals in Austria: a retrospective study

Development and validation of random-forest based federated learning algorithms for delirium prediction using electronic medical records from eleven hospitals in Austria: a retrospective study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Development and validation of random-forest based federated learning algorithms for delirium prediction using electronic medical records from eleven hospitals in Austria: a retrospective study Sai Pavan Kumar Veeranki This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-2970317/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Jan, 2026 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted 7 You are reading this latest preprint version Abstract Background Machine learning models have shown great potential in preventive medicine but require large datasets, which is a challenge due to strict privacy regulations in the healthcare sector. Federated learning is an approach that enables collaboration between institutions while preserving data privacy. The focus today in research is on developing federated learning methods using artificial neural networks. In this study, we aimed to contribute federated learning modelling methods applied for random forests with a use case of predicting delirium in hospitalised patients using data from multiple hospitals. Methods We collected data from 11 hospitals, including 29,479 patients and 627 features. We trained random forest models with each hospital’s data and a general model using all hospitals data. We developed federated learning models by averaging the predictions of the individual hospital models, with different schemes based on the number of samples, positive cases, minority cases and maximum possible diversity and evaluated the models using area under the receiver operating characteristic curve (AUROC) as a performance measure. Results The general model outperformed all the other models with an AUROC of 0.854 [0.849-0.860]. Models trained on data from single hospitals varied in performance with AUROC from 0.626 to 0.828. Models from hospitals with large datasets performed better than that of small hospitals. The general model outperformed all the other models with an AUROC of 0.854. Federated learning models performed better than individual models. Unweighted averaging performed worst with an AUROC of 0.793 [0.782-0.805]. Among the weighted averaging schemes, the number of positive cases performed the best with an AUROC of 0.843 [0.838-0.846], followed by minority class (AUROC=0.840 [0.836-0.845]), maximum possible diversity (AUROC=0.836 [0.830-0.841]) and number of samples (AUROC=0.830 [0.819-0.841]). Conclusions Results suggest that federated learning models can perform better than hospital-specific models in some cases, especially hospitals with limited data. In case of datasets of different size, we suggest weighted averaging based on the number of samples. If the datasets are class imbalanced, maximum possible diversity should also be considered. Additionally, federated learning models are consistent and stable in performance compared to hospital specific models. Full Text Supplementary Files supplementarymaterial.docx Cite Share Download PDF Status: Published Journal Publication published 14 Jan, 2026 Read the published version in BMC Medical Informatics and Decision Making → Version 1 posted Editorial decision: Revision requested 25 Sep, 2024 Reviews received at journal 09 Aug, 2024 Reviewers agreed at journal 24 Jul, 2024 Reviewers agreed at journal 20 Feb, 2024 Reviewers invited by journal 20 Feb, 2024 Submission checks completed at journal 13 Feb, 2024 First submitted to journal 22 Jan, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-2970317","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":291655984,"identity":"619b7d48-f127-4280-8b69-729f50f650cd","order_by":0,"name":"Sai Pavan Kumar Veeranki","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA9ElEQVRIie3RsWrDMBCA4TOCdAl4zVS/wpUuKQT7VWQEzhLoGmihmjwFvDr0ZS4cOIsh6w0Z6jdQ6ZKprVpoKRnUZCtUPwg06OMOBBCL/cEym1g1/ryOgBydQJB+kE37RUIU/fkmanwSuWCbtMvZbfpYlzzr87xJKXlxAZKtSk/66qbdd8QLMWbdajUJTQHyZF0zgswtL5wyKABhshssPL++YfZBpu7BFDtShyCR0hNLiFIRg3COoEfBKSiDJd0ZvJJKb1b9Vk+krKd9gGTNfHjS9zleSnXtDt1dkTbMsgwtBkef4PdM7C/gqOK857FYLPYfegfMS11QcVbzewAAAABJRU5ErkJggg==","orcid":"","institution":"","correspondingAuthor":true,"prefix":"","firstName":"Sai","middleName":"Pavan Kumar","lastName":"Veeranki","suffix":""}],"badges":[],"createdAt":"2023-05-23 08:44:24","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-2970317/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-2970317/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12911-025-03322-y","type":"published","date":"2026-01-14T16:29:26+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":100616092,"identity":"c891b19b-b1b0-43d3-982d-a7df4d9d5ae2","added_by":"auto","created_at":"2026-01-19 17:39:35","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2442410,"visible":true,"origin":"","legend":"","description":"","filename":"manuscriptfinalwithtrackchanges.pdf","url":"https://assets-eu.researchsquare.com/files/rs-2970317/v1_covered_b5d3ed8d-c709-487f-9169-e5384abb4781.pdf"},{"id":54736306,"identity":"b5cbf2b7-33c3-4544-8db8-c53161d719f1","added_by":"auto","created_at":"2024-04-16 04:03:54","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":2473635,"visible":true,"origin":"","legend":"","description":"","filename":"supplementarymaterial.docx","url":"https://assets-eu.researchsquare.com/files/rs-2970317/v1/dda40c9421ee4eb7cec25a40.docx"}],"financialInterests":"","formattedTitle":"Development and validation of random-forest based federated learning algorithms for delirium prediction using electronic medical records from eleven hospitals in Austria: a retrospective study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-2970317/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-2970317/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground\u003c/strong\u003e Machine learning models have shown great potential in preventive medicine but require large datasets, which is a challenge due to strict privacy regulations in the healthcare sector. Federated learning is an approach that enables collaboration between institutions while preserving data privacy. The focus today in research is on developing federated learning methods using artificial neural networks. In this study, we aimed to contribute federated learning modelling methods applied for random forests with a use case of predicting delirium in hospitalised patients using data from multiple hospitals.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e We collected data from 11 hospitals, including 29,479 patients and 627 features. We trained random forest models with each hospital’s data and a general model using all hospitals data. We developed federated learning models by averaging the predictions of the individual hospital models, with different schemes based on the number of samples, positive cases, minority cases and maximum possible diversity and evaluated the models using area under the receiver operating characteristic curve (AUROC) as a performance measure.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e The general model outperformed all the other models with an AUROC of 0.854 [0.849-0.860]. Models trained on data from single hospitals varied in performance with AUROC from 0.626 to 0.828. Models from hospitals with large datasets performed better than that of small hospitals. The general model outperformed all the other models with an AUROC of 0.854. Federated learning models performed better than individual models. Unweighted averaging performed worst with an AUROC of 0.793 [0.782-0.805]. Among the weighted averaging schemes, the number of positive cases performed the best with an AUROC of 0.843 [0.838-0.846], followed by minority class (AUROC=0.840 [0.836-0.845]), maximum possible diversity (AUROC=0.836 [0.830-0.841]) and number of samples (AUROC=0.830 [0.819-0.841]).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions\u003c/strong\u003e Results suggest that federated learning models can perform better than hospital-specific models in some cases, especially hospitals with limited data. In case of datasets of different size, we suggest weighted averaging based on the number of samples. If the datasets are class imbalanced, maximum possible diversity should also be considered. Additionally, federated learning models are consistent and stable in performance compared to hospital specific models.\u003c/p\u003e","manuscriptTitle":"Development and validation of random-forest based federated learning algorithms for delirium prediction using electronic medical records from eleven hospitals in Austria: a retrospective study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-04-16 04:03:49","doi":"10.21203/rs.3.rs-2970317/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-09-25T15:40:04+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-08-09T13:09:33+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"117034541675424483207478911235391076391","date":"2024-07-24T22:08:13+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"2ceda65e-2f9e-4d98-94ab-66d3c16a2f5c","date":"2024-02-20T15:16:18+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-02-20T15:06:38+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-02-13T11:53:24+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Informatics and Decision Making","date":"2024-01-22T12:42:53+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-informatics-and-decision-making","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"midm","sideBox":"Learn more about [BMC Medical Informatics and Decision Making](http://bmcmedinformdecismak.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/midm/default.aspx","title":"BMC Medical Informatics and Decision Making","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"a7b27cc0-c8bc-489a-9243-5a224f86ef0f","owner":[],"postedDate":"April 16th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2026-01-19T17:04:47+00:00","versionOfRecord":{"articleIdentity":"rs-2970317","link":"https://doi.org/10.1186/s12911-025-03322-y","journal":{"identity":"bmc-medical-informatics-and-decision-making","isVorOnly":false,"title":"BMC Medical Informatics and Decision Making"},"publishedOn":"2026-01-14 16:29:26","publishedOnDateReadable":"January 14th, 2026"},"versionCreatedAt":"2024-04-16 04:03:49","video":"","vorDoi":"10.1186/s12911-025-03322-y","vorDoiUrl":"https://doi.org/10.1186/s12911-025-03322-y","workflowStages":[]},"version":"v1","identity":"rs-2970317","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-2970317","identity":"rs-2970317","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}