FairPlay: Improving Medical Machine Learning Models with Generative Balancing for Equity and Excellence

FairPlay: Improving Medical Machine Learning Models with Generative Balancing for Equity and Excellence | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article FairPlay: Improving Medical Machine Learning Models with Generative Balancing for Equity and Excellence Brandon Theodorou, Benjamin Danek, Venkat Tummala, Shivam Pankaj Kumar, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5252769/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 14 Feb, 2025 Read the published version in npj Digital Medicine → Version 1 posted 13 You are reading this latest preprint version Abstract Understanding and addressing the innate complexities of applying machine learning to clinical outcome prediction requires a deep knowledge of the unique challenges inherent in this field. These challenges are particularly pronounced due to imbalanced datasets and the sensitive nature of tasks, especially in scenarios where critical outcomes are rare and ensuring equitable treatment across diverse patient groups is paramount. Despite attempts to improve these models and to bridge the divide in performance between different gender, racial, and ethnic groups, the disparity in representation, exacerbated by the overall scarcity of positive labels, contributes to biased predictions and decisions, further perpetuating health disparities. This paper proposes a powerful approach using synthetic patient data, FairPlay, offering a dual solution: enhancing overall algorithmic performance while mitigating bias by bolstering un-derrepresented populations. By generating realistic, anonymous synthetic data echoing real patient characteristics through an advanced , large language model-based approach, we not only improve representation without compromising privacy but also improve the general performance of algorithms by exposing them to a richer variety of patient profiles. We demonstrate the effectiveness of this approach through a variety of experiments on multiple datasets. We showcase that FairPlay can improve the performance of mortality prediction both overall and across multiple different groups. Specifically, we find that FairPlay can improve F1 Score averaged across a suite of downstream models by up to 21% without accessing any additional data or changing any of the downstream model training regimens. It achieves this across every subgroup while simultaneously shrinking the gap between different subgroup performances, as demonstrated through the universal improvement of a pair of fairness metrics across four different experimental setups. Health sciences/Risk factors Physical sciences/Mathematics and computing/Computer science Full Text Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 14 Feb, 2025 Read the published version in npj Digital Medicine → Version 1 posted Editorial decision: Revision requested 17 Nov, 2024 Reviews received at journal 11 Nov, 2024 Reviews received at journal 11 Nov, 2024 Reviews received at journal 08 Nov, 2024 Reviews received at journal 07 Nov, 2024 Reviewers agreed at journal 29 Oct, 2024 Reviewers agreed at journal 27 Oct, 2024 Reviewers agreed at journal 27 Oct, 2024 Reviewers agreed at journal 27 Oct, 2024 Reviewers invited by journal 27 Oct, 2024 Editor assigned by journal 14 Oct, 2024 Submission checks completed at journal 14 Oct, 2024 First submitted to journal 12 Oct, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5252769","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":379055137,"identity":"9d2db039-58d0-4bb6-859e-c3841ca0ca40","order_by":0,"name":"Brandon Theodorou","email":"","orcid":"","institution":"University of Illinois Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Brandon","middleName":"","lastName":"Theodorou","suffix":""},{"id":379055138,"identity":"1da6b921-98b8-414c-aad8-b9055c11dc68","order_by":1,"name":"Benjamin Danek","email":"","orcid":"","institution":"University of Illinois Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Benjamin","middleName":"","lastName":"Danek","suffix":""},{"id":379055139,"identity":"acd7160d-de3d-41ec-afdd-509804a9eac2","order_by":2,"name":"Venkat Tummala","email":"","orcid":"","institution":"University of Illinois Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Venkat","middleName":"","lastName":"Tummala","suffix":""},{"id":379055141,"identity":"10f1dfc8-5b62-4cae-b0db-dfc7d09673d0","order_by":3,"name":"Shivam Pankaj Kumar","email":"","orcid":"","institution":"University of Illinois Urbana-Champaign","correspondingAuthor":false,"prefix":"","firstName":"Shivam","middleName":"Pankaj","lastName":"Kumar","suffix":""},{"id":379055142,"identity":"a1fcc52f-6074-4c05-bc9b-887fe8edf50c","order_by":4,"name":"Bradley Malin","email":"","orcid":"","institution":"Vanderbilt University","correspondingAuthor":false,"prefix":"","firstName":"Bradley","middleName":"","lastName":"Malin","suffix":""},{"id":379055143,"identity":"b6a0e634-7b9a-4ed1-9a11-a8d7655ee5c9","order_by":5,"name":"Jimeng Sun","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzUlEQVRIiWNgGAWjYBACxmYwZcPDx87YwMBgAOIkEKUljYeNmVgtUHCYgY0ZziGghbmdO/Fxwa/zMmzMzG0PPhTUMfCz5xgQcBjvZuOZfbdBDms3nGFwmEGy5w1BLdukeXvAWtqkeQwOMBjcIGzL9t+8PedgWuoY7InQso2Z58cBmBZmBgMJIvwizduQDNYiCfQLj8SZZwV4tRj2n934meePnT0/e/sziQ9/6uT425M34NfSALKqDSHAg1c5CMiDyT8E1Y2CUTAKRsFIBgA2TjqLD5q+9wAAAABJRU5ErkJggg==","orcid":"","institution":"University of Illinois Urbana-Champaign","correspondingAuthor":true,"prefix":"","firstName":"Jimeng","middleName":"","lastName":"Sun","suffix":""}],"badges":[],"createdAt":"2024-10-12 18:23:07","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5252769/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5252769/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41746-025-01438-z","type":"published","date":"2025-02-14T15:57:06+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":76487630,"identity":"1db7f9ee-666c-4708-b1c0-3462f49b8daf","added_by":"auto","created_at":"2025-02-17 16:10:12","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1191966,"visible":true,"origin":"","legend":"","description":"","filename":"SyntheticFairness.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5252769/v1_covered_2c10daa0-10b7-4936-93fb-a7cc3a895192.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"FairPlay: Improving Medical Machine Learning Models with Generative Balancing for Equity and Excellence","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-5252769/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5252769/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eUnderstanding and addressing the innate complexities of applying machine learning to clinical outcome prediction requires a deep knowledge of the unique challenges inherent in this field. These challenges are particularly pronounced due to imbalanced datasets and the sensitive nature of tasks, especially in scenarios where critical outcomes are rare and ensuring equitable treatment across diverse patient groups is paramount. Despite attempts to improve these models and to bridge the divide in performance between different gender, racial, and ethnic groups, the disparity in representation, exacerbated by the overall scarcity of positive labels, contributes to biased predictions and decisions, further perpetuating health disparities.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThis paper proposes a powerful approach using synthetic patient data, FairPlay, offering a dual solution: enhancing overall algorithmic performance while mitigating bias by bolstering un-derrepresented populations. By generating realistic, anonymous synthetic data echoing real patient characteristics through an advanced , large language model-based approach, we not only improve representation without compromising privacy but also improve the general performance of algorithms by exposing them to a richer variety of patient profiles.\u003c/p\u003e\n\u003cp\u003eWe demonstrate the effectiveness of this approach through a variety of experiments on multiple datasets. We showcase that FairPlay can improve the performance of mortality prediction both overall and across multiple different groups. Specifically, we find that FairPlay can improve F1 Score averaged across a suite of downstream models by up to 21% without accessing any additional data or changing any of the downstream model training regimens. It achieves this across every subgroup while simultaneously shrinking the gap between different subgroup performances, as demonstrated through the universal improvement of a pair of fairness metrics across four different experimental setups.\u003c/p\u003e","manuscriptTitle":"FairPlay: Improving Medical Machine Learning Models with Generative Balancing for Equity and Excellence","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-12-13 09:19:48","doi":"10.21203/rs.3.rs-5252769/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-11-17T12:21:20+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-11T21:24:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-11T17:03:43+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-09T04:40:41+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-11-07T13:47:36+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"122766718821773157571470011354244544450","date":"2024-10-29T08:03:31+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"247837535680878855332596577064754019149","date":"2024-10-28T01:27:03+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"193206911185703621815294333244011203146","date":"2024-10-27T06:39:25+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"43357522699752656915300180409911803221","date":"2024-10-27T05:01:55+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-10-27T04:54:57+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-10-14T15:57:19+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-10-14T04:47:37+00:00","index":"","fulltext":""},{"type":"submitted","content":"npj Digital Medicine","date":"2024-10-12T18:17:26+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"npj-digital-medicine","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"npjdigitalmed","sideBox":"Learn more about [npj Digital Medicine](http://www.nature.com/npjdigitalmed/)","snPcode":"41746","submissionUrl":"https://submission.springernature.com/new-submission/41746/3","title":"npj Digital Medicine","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"NPJ","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"ef312b4a-c75f-4d83-99a1-e440aa946025","owner":[],"postedDate":"December 13th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":40362963,"name":"Health sciences/Risk factors"},{"id":40362964,"name":"Physical sciences/Mathematics and computing/Computer science"}],"tags":[],"updatedAt":"2025-02-17T16:04:08+00:00","versionOfRecord":{"articleIdentity":"rs-5252769","link":"https://doi.org/10.1038/s41746-025-01438-z","journal":{"identity":"npj-digital-medicine","isVorOnly":false,"title":"npj Digital Medicine"},"publishedOn":"2025-02-14 15:57:06","publishedOnDateReadable":"February 14th, 2025"},"versionCreatedAt":"2024-12-13 09:19:48","video":"","vorDoi":"10.1038/s41746-025-01438-z","vorDoiUrl":"https://doi.org/10.1038/s41746-025-01438-z","workflowStages":[]},"version":"v1","identity":"rs-5252769","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5252769","identity":"rs-5252769","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}