Joint Models in Big Data: Simulation-Based Guidelines for Required Data Quality in Longitudinal Electronic Health Records

Joint Models in Big Data: Simulation-Based Guidelines for Required Data Quality in Longitudinal Electronic Health Records | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Method Article Joint Models in Big Data: Simulation-Based Guidelines for Required Data Quality in Longitudinal Electronic Health Records Berit Hunsdieck, Christian Bender, Katja Ickstadt, Johanna Mielke This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6031358/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 13 May, 2025 Read the published version in BioData Mining → Version 1 posted 7 You are reading this latest preprint version Abstract Background: Over the past decade an increase in usage of electronic health data (EHR) by office-based physicians and hospitals has been reported. However, these data types come with challenge regarding completeness and data quality and it is, especially for more complex models, unclear how these characteristics influence the performance. Methods: In this paper, we focus on joint models which combines longitudinal modelling with survival modelling to incorporate all available information. The aim of this paper is to establish simulation-based guidelines for the necessary quality of longitudinal EHR data so that joint models perform better than cox models. We conducted an extensive simulation study by systematically and transparently varying different characteristics of data quality, e.g., measurement frequency, noise, and heterogeneity between patients. We apply the joint models and evaluate their performance relative to traditional Cox survival modelling techniques. Results: Key findings suggest that biomarker changes before disease onset must be consistent within similar patient groups. With increasing noise and a higher measurement density, the joint model surpasses the traditional Cox regression model in terms of model performance.We illustrate the usefulness and limitations of the guidelines with two real-world examples, namely the influence of serum bilirubin on primary biliary liver cirrhosis and the influence of the estimated glomerular filtration rate on chronic kidney disease. Joint Modelling Longitudinal Data Application Primary Care Data Simulation Study Chronic Kidney Disease Full Text Additional Declarations Competing interest reported. Berit Hunsdieck, Christian Bender and Johanna Mielke are employees of Bayer AG. Cite Share Download PDF Status: Published Journal Publication published 13 May, 2025 Read the published version in BioData Mining → Version 1 posted Editorial decision: Revision requested 28 Mar, 2025 Reviews received at journal 24 Mar, 2025 Reviewers agreed at journal 14 Mar, 2025 Reviewers invited by journal 03 Mar, 2025 Editor assigned by journal 19 Feb, 2025 Submission checks completed at journal 19 Feb, 2025 First submitted to journal 14 Feb, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6031358","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Method Article","associatedPublications":[],"authors":[{"id":433397925,"identity":"8c9b70ce-cc97-4446-b581-6043e26281d6","order_by":0,"name":"Berit Hunsdieck","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABA0lEQVRIiWNgGAWjYBACAyA+AEZANpBMkDMgSssBJC3GRGlhgGkB4oTEDYS0mLP3Pjz8geFO4nb+wxsPfKhJS9/OfsbwcQWDnZxuA3Ytlj3HDYAOe5a4c0ZawcEZx3Jyd/bkGBueYUg2NjuAw2E30kB+OZy44QaPwWHehorcDQdyzCQbGA4kbsOl5f4zqJbzZwwO/22oSDc4/8b8J14tN9igWg7kGBxmbMhJMLiRY8aIT4tlD9BhQPONN9wA+qXnWJrhzhnPiiUbDHD7xZz9GPOHiorDshvOH9784UdNsrw5f/LGjw0VdnK4tECdh8LjMEAXIQjYH5CmfhSMglEwCoY7AAD+bG6ckC7x4QAAAABJRU5ErkJggg==","orcid":"","institution":"Bayer (Germany)","correspondingAuthor":true,"prefix":"","firstName":"Berit","middleName":"","lastName":"Hunsdieck","suffix":""},{"id":433397926,"identity":"e35d527a-37c4-4282-b354-29e4ef08df17","order_by":1,"name":"Christian Bender","email":"","orcid":"","institution":"Bayer (Germany)","correspondingAuthor":false,"prefix":"","firstName":"Christian","middleName":"","lastName":"Bender","suffix":""},{"id":433397927,"identity":"e1d0fcb9-3679-4df5-adb0-ba5823d9fc64","order_by":2,"name":"Katja Ickstadt","email":"","orcid":"","institution":"TU Dortmund University","correspondingAuthor":false,"prefix":"","firstName":"Katja","middleName":"","lastName":"Ickstadt","suffix":""},{"id":433397928,"identity":"27d6d8d7-6d8b-43bd-aaa3-27fbc457b67a","order_by":3,"name":"Johanna Mielke","email":"","orcid":"","institution":"Bayer (Germany)","correspondingAuthor":false,"prefix":"","firstName":"Johanna","middleName":"","lastName":"Mielke","suffix":""}],"badges":[],"createdAt":"2025-02-14 14:23:10","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6031358/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6031358/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s13040-025-00450-z","type":"published","date":"2025-05-13T15:57:04+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":83067816,"identity":"b3a6a794-ee32-4ae6-a9ef-30cfe6494e53","added_by":"auto","created_at":"2025-05-19 16:06:37","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":563011,"visible":true,"origin":"","legend":"","description":"","filename":"SpringerNatureLaTeXTemplate1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6031358/v1_covered_70a17784-4970-45a3-8147-3d9cd317ecc6.pdf"}],"financialInterests":"Competing interest reported. Berit Hunsdieck, Christian Bender and Johanna Mielke are employees of Bayer AG.","formattedTitle":"Joint Models in Big Data: Simulation-Based Guidelines for Required Data Quality in Longitudinal Electronic Health Records","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"biodata-mining","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bidm","sideBox":"Learn more about [BioData Mining](http://biodatamining.biomedcentral.com/)","snPcode":"13040","submissionUrl":"https://submission.nature.com/new-submission/13040/3","title":"BioData Mining","twitterHandle":"@BioMedCentral","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Joint Modelling, Longitudinal Data Application, Primary Care Data, Simulation Study, Chronic Kidney Disease","lastPublishedDoi":"10.21203/rs.3.rs-6031358/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6031358/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Background: Over the past decade an increase in usage of electronic health data (EHR) by office-based physicians and hospitals has been reported. However, these data types come with challenge regarding completeness and data quality and it is, especially for more complex models, unclear how these characteristics influence the performance.\nMethods: In this paper, we focus on joint models which combines longitudinal modelling with survival modelling to incorporate all available information. The aim of this paper is to establish simulation-based guidelines for the necessary quality of longitudinal EHR data so that joint models perform better than cox models. We conducted an extensive simulation study by systematically and transparently varying different characteristics of data quality, e.g., measurement frequency, noise, and heterogeneity between patients. We apply the joint models and evaluate their performance relative to traditional Cox survival modelling techniques.\nResults: Key findings suggest that biomarker changes before disease onset must be consistent within similar patient groups. With increasing noise and a higher measurement density, the joint model surpasses the traditional Cox regression model in terms of model performance.We illustrate the usefulness and limitations of the guidelines with two real-world examples, namely the influence of serum bilirubin on primary biliary liver cirrhosis and the influence of the estimated glomerular filtration rate on chronic kidney disease. ","manuscriptTitle":"Joint Models in Big Data: Simulation-Based Guidelines for Required Data Quality in Longitudinal Electronic Health Records","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-03-27 04:24:01","doi":"10.21203/rs.3.rs-6031358/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-03-28T17:10:55+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-03-24T20:43:36+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"64221221790195210330083154016905203841","date":"2025-03-14T19:03:03+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-03-03T23:09:03+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-02-19T06:46:16+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-02-19T06:44:59+00:00","index":"","fulltext":""},{"type":"submitted","content":"BioData Mining","date":"2025-02-14T14:08:53+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"biodata-mining","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"bidm","sideBox":"Learn more about [BioData Mining](http://biodatamining.biomedcentral.com/)","snPcode":"13040","submissionUrl":"https://submission.nature.com/new-submission/13040/3","title":"BioData Mining","twitterHandle":"@BioMedCentral","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"em","reportingPortfolio":"BMC/SO AJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"efcc0985-86e1-43a5-af2c-2e1154649256","owner":[],"postedDate":"March 27th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-05-19T16:00:44+00:00","versionOfRecord":{"articleIdentity":"rs-6031358","link":"https://doi.org/10.1186/s13040-025-00450-z","journal":{"identity":"biodata-mining","isVorOnly":false,"title":"BioData Mining"},"publishedOn":"2025-05-13 15:57:04","publishedOnDateReadable":"May 13th, 2025"},"versionCreatedAt":"2025-03-27 04:24:01","video":"","vorDoi":"10.1186/s13040-025-00450-z","vorDoiUrl":"https://doi.org/10.1186/s13040-025-00450-z","workflowStages":[]},"version":"v1","identity":"rs-6031358","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6031358","identity":"rs-6031358","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}