TRAPOD: A Transformer Architecture Exploits Intraoperative Temporal Dynamics Improving the Prediction of Postoperative Delirium

TRAPOD: A Transformer Architecture Exploits Intraoperative Temporal Dynamics Improving the Prediction of Postoperative Delirium | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article TRAPOD: A Transformer Architecture Exploits Intraoperative Temporal Dynamics Improving the Prediction of Postoperative Delirium Niklas Giesa, Maria Sekutowicz, Kerstin Rubarth, Claudia Spies, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4349783/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 27 Nov, 2024 Read the published version in Communications Medicine → Version 1 posted You are reading this latest preprint version Abstract Motivation. Patients who experienced postoperative delirium (POD) are at higher risk of poor outcomes like dementia or death. Previous machine learning (ML) models predicting POD mostly relied on time-aggregated features. Objective. We assessed the potential of temporal patterns in clinical parameters during surgeries to predict POD. Methods. Long short-term memory (LSTM) and transformer models, directly consuming time series, were compared to multi-layer perceptrons (MLPs) trained on time-aggregated features. We also fitted hybrid models, fusing either LSTM or transformer models with MLPs. Univariate Spearman’s rank correlations and linear mixed-effect models establish the importance of individual features that we compared to transformers’ attention weights. Results. Best performance was achieved by a transformer architecture ingesting 30 minutes of intraoperative parameter sequences. Systolic invasive blood pressure and given opioids marked the most important input variables, in line with univariate feature importances. Conclusion. Intraoperative temporal dynamics of clinical parameters, exploited by a tailored transformer architecture named TRAPOD, are critical for the accurate prediction of POD. Health sciences/Health care/Prognosis Health sciences/Neurology/Neurological disorders Clinical Data Science Machine Learning Time Series Transformer Postoperative Delirium Prediction Full Text Additional Declarations There is NO Competing Interest. Supplementary Files supplementAsubmissionnature30042024.pdf supplementBsubmissionnature30042024.xlsx rsn.pdf Reporting Summary Cite Share Download PDF Status: Published Journal Publication published 27 Nov, 2024 Read the published version in Communications Medicine → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4349783","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":304559383,"identity":"964142fb-e02a-4121-9328-e29b46623665","order_by":0,"name":"Niklas Giesa","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABBUlEQVRIie3NsWrDMBCA4RMCeXHJKjDFr5AQSDC49qsoGDq5a6YMgoC8NJ099GFkBO5S0rVDh5pAu3QwdDHU0MomBDrI7liI/kG6A30IwGb7j2GCZHeT7qhBz/i0GgmcCMp7gkcIHEnP3X4eIUuHMNm0EN05WfUZbl78ZTYpXmEdGkmwJbLYCUiE+zj30vJtdq8wnsL+2kimyuHygkNCaAreDVcox5hQJNQgKdpWE//98BVwFWviNEh8DxAilUsgIhQWHuJq1f0CSMghwtSloIy46SK4LVWiyZyyfWImT+Ws+mjD2M8eDs/NRl3lk6Kq63VkJMfoiv/a2RjQxX94Y7PZbOfaD5B2UTy18Wl2AAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-0808-3966","institution":"Charit\u0026#x00E9","correspondingAuthor":true,"prefix":"","firstName":"Niklas","middleName":"","lastName":"Giesa","suffix":""},{"id":304559384,"identity":"c408b078-3aaa-4d3c-89e5-f303f631f259","order_by":1,"name":"Maria Sekutowicz","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Maria","middleName":"","lastName":"Sekutowicz","suffix":""},{"id":304559385,"identity":"e1a0af51-2f3f-4d94-94db-84dc5606af65","order_by":2,"name":"Kerstin Rubarth","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Kerstin","middleName":"","lastName":"Rubarth","suffix":""},{"id":304559386,"identity":"20b0f0d9-331d-4c82-a322-bb4a490427a2","order_by":3,"name":"Claudia Spies","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Claudia","middleName":"","lastName":"Spies","suffix":""},{"id":304559387,"identity":"436b0a0a-cf30-47f7-9d26-3da8ac3066f2","order_by":4,"name":"Felix Balzer","email":"","orcid":"https://orcid.org/0000-0003-1575-2056","institution":"Charite - Universitätsmedizin Berlin","correspondingAuthor":false,"prefix":"","firstName":"Felix","middleName":"","lastName":"Balzer","suffix":""},{"id":304559388,"identity":"fa8bb628-45c0-4469-b56e-8a8065b86cd9","order_by":5,"name":"Stefan Haufe","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Stefan","middleName":"","lastName":"Haufe","suffix":""},{"id":304559389,"identity":"b63525bf-e4a6-4326-ac3b-47b1e8749036","order_by":6,"name":"Sebastian Daniel Boie","email":"","orcid":"","institution":"","correspondingAuthor":false,"prefix":"","firstName":"Sebastian","middleName":"Daniel","lastName":"Boie","suffix":""}],"badges":[],"createdAt":"2024-04-30 14:36:06","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4349783/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4349783/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s43856-024-00681-x","type":"published","date":"2024-11-27T05:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":70085082,"identity":"326a7018-86be-40f1-8e85-b455a72ef94f","added_by":"auto","created_at":"2024-11-28 08:05:53","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":3056536,"visible":true,"origin":"","legend":"","description":"","filename":"maintextsubmissionnature30042024.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4349783/v1_covered_3b1d9442-135c-482c-96d4-01b99c3488b2.pdf"},{"id":57459892,"identity":"b85e8873-a59a-4550-a8e2-ae763639c8df","added_by":"auto","created_at":"2024-05-31 02:40:19","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":3309645,"visible":true,"origin":"","legend":"","description":"","filename":"supplementAsubmissionnature30042024.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4349783/v1/f71a5eaf5ad5213de881f8e3.pdf"},{"id":57459482,"identity":"2997ff17-27a8-4bbb-8fa8-1f0ace5df39c","added_by":"auto","created_at":"2024-05-31 02:32:19","extension":"xlsx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":140714,"visible":true,"origin":"","legend":"","description":"","filename":"supplementBsubmissionnature30042024.xlsx","url":"https://assets-eu.researchsquare.com/files/rs-4349783/v1/3dc581a5d96a565e415951c9.xlsx"},{"id":57459483,"identity":"36c8929d-7127-42f1-912b-d8b9b853d0fc","added_by":"auto","created_at":"2024-05-31 02:32:19","extension":"pdf","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":2524654,"visible":true,"origin":"","legend":"Reporting Summary","description":"","filename":"rsn.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4349783/v1/59a5d75534f72cc60fa75273.pdf"}],"financialInterests":"There is \u003cb\u003eNO\u003c/b\u003e Competing Interest.","formattedTitle":"TRAPOD: A Transformer Architecture Exploits Intraoperative Temporal Dynamics Improving the Prediction of Postoperative Delirium","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":" Clinical Data Science, Machine Learning, Time Series, Transformer, Postoperative Delirium Prediction","lastPublishedDoi":"10.21203/rs.3.rs-4349783/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4349783/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eMotivation.\u003c/strong\u003e Patients who experienced postoperative delirium (POD) are at higher risk of poor outcomes like dementia or death. Previous machine learning (ML) models predicting POD mostly relied on time-aggregated features.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eObjective.\u003c/strong\u003e We assessed the potential of temporal patterns in clinical parameters during surgeries to predict POD.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods.\u003c/strong\u003e Long short-term memory (LSTM) and transformer models, directly consuming time series, were compared to multi-layer perceptrons (MLPs) trained on time-aggregated features. We also fitted hybrid models, fusing either LSTM or transformer models with MLPs. Univariate Spearman’s rank correlations and linear mixed-effect models establish the importance of individual features that we compared to transformers’ attention weights.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults. \u003c/strong\u003eBest performance was achieved by a transformer architecture ingesting 30 minutes of intraoperative parameter sequences. Systolic invasive blood pressure and given opioids marked the most important input variables, in line with univariate feature importances.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion.\u003c/strong\u003e Intraoperative temporal dynamics of clinical parameters, exploited by a tailored transformer architecture named TRAPOD, are critical for the accurate prediction of POD.\u003c/p\u003e","manuscriptTitle":"TRAPOD: A Transformer Architecture Exploits Intraoperative Temporal Dynamics Improving the Prediction of Postoperative Delirium","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-31 02:32:15","doi":"10.21203/rs.3.rs-4349783/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"communications-medicine","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"commsmed","sideBox":"Learn more about [Communications Medicine](http://www.nature.com/commsmed)","snPcode":"43856","submissionUrl":"https://mts-commsmed.nature.com/cgi-bin/main.plex","title":"Communications Medicine","twitterHandle":"@commsmedicine","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Communications Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"7246111c-33f5-4e6a-b56d-6590a7bc6f5c","owner":[],"postedDate":"May 31st, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":32152298,"name":"Health sciences/Health care/Prognosis"},{"id":32152299,"name":"Health sciences/Neurology/Neurological disorders"}],"tags":[],"updatedAt":"2024-11-28T08:05:41+00:00","versionOfRecord":{"articleIdentity":"rs-4349783","link":"https://doi.org/10.1038/s43856-024-00681-x","journal":{"identity":"communications-medicine","isVorOnly":false,"title":"Communications Medicine"},"publishedOn":"2024-11-27 05:00:00","publishedOnDateReadable":"November 27th, 2024"},"versionCreatedAt":"2024-05-31 02:32:15","video":"","vorDoi":"10.1038/s43856-024-00681-x","vorDoiUrl":"https://doi.org/10.1038/s43856-024-00681-x","workflowStages":[]},"version":"v1","identity":"rs-4349783","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4349783","identity":"rs-4349783","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}