Predicting IVF live birth probabilities using machine learning, center-specific models: validation results and potential benefits over national registry-based models.

Predicting IVF live birth probabilities using machine learning, center-specific models: validation results and potential benefits over national registry-based models. | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Predicting IVF live birth probabilities using machine learning, center-specific models: validation results and potential benefits over national registry-based models. Mylene Yao, Elizabeth Nguyen, Matthew Retzloff, L. Gago, John Nichols, and 10 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5140601/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 17 Apr, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Abstract Ongoing improvement of pretreatment live birth prognostication for in vitro fertilization (IVF) is critical for informing fertility patients' treatment decisions, advocating for IVF coverage and supporting value-based IVF care. The US national registry Society for Assisted Reproductive Technology (SART) IVF live birth prediction (LBP) model (SART model) has been widely adopted for its prognostic support without external validation or utilization studies. We conducted a retrospective model validation study to compare the IVF LBP performance of machine learning, center-specific (MLCS) models versus the SART model in 6 unrelated US fertility centers using their respective center-specific test sets comprising an aggregate of 4,635 patients' first-IVF cycle data. Compared to the SART model, MLCS2 showed higher median Precision Recall AUC at 0.75 (IQR 0.73, 0.77) vs. 0.69 (IQR 0.68, 0.71), p<0.05 and higher median F1 Score across LBP thresholds. Further, MLCS1 showed no evidence of data drift when validated using out-of-time test data from a later period. Reclassification analysis showed that MLCS2 models assigned more appropriate and higher IVF LBPs compared to the SART model, which underestimated patient prognoses (continuous net reclassification index: 18.3%, p<0.0001). Overall, MLCS2 and SART models assigned 30% of patients to differential prognostic groups, with MLCS2 assigning 26% of patients to a higher LBP category compared to the SART model. Importantly, MLCS2 models identified 11% of patients to have LBP ≥ 75%, whereas the SART model detected none. This group had a live birth rate of 81%. We recommend testing a larger sample of fertility centers to further evaluate MLCS model benefits and limitations. Health sciences/Health care/Prognosis/Pregnancy outcome Scientific community and society/Business and industry/Technology Health sciences/Medical research/Outcomes research Health sciences/Diseases/Reproductive disorders/Infertility Health sciences/Health care/Health policy live birth probability IVF live birth prediction artificial intelligence machine learning SART fertility prognosis Full Text Additional Declarations Yes there is potential Competing Interest. M Yao is employed as CEO by Univfy Inc. and is board director, shareholder and stock optionee of Univfy; she is inventor or co-inventor on Univfy's issued and pending patents and receives payment from patent licensor (Stanford University). ET Nguyen, T Swanson, X Chen are employed by and received stock options from Univfy Inc. M Retzloff performs paid consulting work as Nexplanon trainer for Organon and is Treasurer for the Society for Reproductive Technology (SART). Cite Share Download PDF Status: Published Journal Publication published 17 Apr, 2025 Read the published version in Nature Communications → Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5140601","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":359012544,"identity":"e345263b-8e83-4a70-8fa3-6532f1208a75","order_by":0,"name":"Mylene Yao","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5UlEQVRIiWNgGAWjYFAC5gYY4wCQkJAhQgsjTAtbAkgLDylaeAzAJEENujMSGx8XVDAk9rf3fH51o8aCh4H98NEN+LSY3UhsNp5xhiFxxpmz26xzjgEdxpOWdoOAljZp3jaGxIYbuduMc9iAWiR4zAhpaf/N+48hcf6NnGfGOf+I09LGzNvAkLjhRg7z49w2YrScedgszXNMwnjjmWNmzLl9EjxsBP1yPPngZ54aG9l5x5sff875VifHz374GF4tDAIJIFLCsQEYlxIgJhte5SDAfwBM2QMx8weCqkfBKBgFo2BEAgC9UUqrc3E6fgAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0009-0001-8381-6149","institution":"Univfy Inc","correspondingAuthor":true,"prefix":"","firstName":"Mylene","middleName":"","lastName":"Yao","suffix":""},{"id":359012545,"identity":"3a6022ac-8353-4fcd-8940-34d515768bad","order_by":1,"name":"Elizabeth Nguyen","email":"","orcid":"","institution":"Univfy Inc.","correspondingAuthor":false,"prefix":"","firstName":"Elizabeth","middleName":"","lastName":"Nguyen","suffix":""},{"id":359012546,"identity":"b7b468d1-e6c0-4b5b-b0e6-eda1493ebd0e","order_by":2,"name":"Matthew Retzloff","email":"","orcid":"","institution":"Fertility Center of San Antonio","correspondingAuthor":false,"prefix":"","firstName":"Matthew","middleName":"","lastName":"Retzloff","suffix":""},{"id":359012547,"identity":"699b8f6b-923c-4ca8-a64a-a44dc0a99e3f","order_by":3,"name":"L. Gago","email":"","orcid":"","institution":"Gago Center for Fertility","correspondingAuthor":false,"prefix":"","firstName":"L.","middleName":"","lastName":"Gago","suffix":""},{"id":359012548,"identity":"ec7b3316-d131-4f63-94a9-e7e3c15709ca","order_by":4,"name":"John Nichols","email":"","orcid":"","institution":"Piedmont Reproductive Endocrinology Group","correspondingAuthor":false,"prefix":"","firstName":"John","middleName":"","lastName":"Nichols","suffix":""},{"id":359012549,"identity":"3c482416-d8c4-4781-b637-9f7a6d2bef75","order_by":5,"name":"John Payne","email":"","orcid":"","institution":"Piedmont Reproductive Endocrinology Group","correspondingAuthor":false,"prefix":"","firstName":"John","middleName":"","lastName":"Payne","suffix":""},{"id":359012550,"identity":"81429255-075d-4be5-a731-2aa360ba2c5b","order_by":6,"name":"Barry Ripps","email":"","orcid":"","institution":"NewLIFE Fertility","correspondingAuthor":false,"prefix":"","firstName":"Barry","middleName":"","lastName":"Ripps","suffix":""},{"id":359012551,"identity":"383a6744-0302-495e-98a3-9a80e01bfafc","order_by":7,"name":"Michael Opsahl","email":"","orcid":"","institution":"Poma Fertility","correspondingAuthor":false,"prefix":"","firstName":"Michael","middleName":"","lastName":"Opsahl","suffix":""},{"id":359012552,"identity":"76f7d1d7-7008-44ca-9340-ddfb482c9faa","order_by":8,"name":"Jeremy Groll","email":"","orcid":"","institution":"SpringCreek Fertility","correspondingAuthor":false,"prefix":"","firstName":"Jeremy","middleName":"","lastName":"Groll","suffix":""},{"id":359012553,"identity":"0cf5e8e1-10a4-4ec4-8531-044c781bdb9e","order_by":9,"name":"Ronald Beesley","email":"","orcid":"","institution":"Poma Fertility","correspondingAuthor":false,"prefix":"","firstName":"Ronald","middleName":"","lastName":"Beesley","suffix":""},{"id":359012554,"identity":"dd2726a2-7054-45ac-b0c6-33e12e84ab28","order_by":10,"name":"Gregory Neal","email":"","orcid":"","institution":"Fertility Center of San Antonio","correspondingAuthor":false,"prefix":"","firstName":"Gregory","middleName":"","lastName":"Neal","suffix":""},{"id":359012555,"identity":"be6cd7a4-f9c3-4c93-b78e-b558dc4a3106","order_by":11,"name":"Jaye Adams","email":"","orcid":"","institution":"Fertility Center of San Antonio","correspondingAuthor":false,"prefix":"","firstName":"Jaye","middleName":"","lastName":"Adams","suffix":""},{"id":359012556,"identity":"73e4f5bb-8f4b-47d7-b7af-7b11859c8cfd","order_by":12,"name":"Lorie Nowak","email":"","orcid":"","institution":"SpringCreek Fertility","correspondingAuthor":false,"prefix":"","firstName":"Lorie","middleName":"","lastName":"Nowak","suffix":""},{"id":359012557,"identity":"2a675c0a-6203-4f91-84e5-2449adc4d916","order_by":13,"name":"Trevor Swanson","email":"","orcid":"","institution":"Univfy Inc.","correspondingAuthor":false,"prefix":"","firstName":"Trevor","middleName":"","lastName":"Swanson","suffix":""},{"id":359012558,"identity":"30732b68-88ee-4918-8b16-3fafa2f428a8","order_by":14,"name":"Xiaocong Chen","email":"","orcid":"","institution":"Univfy Inc.","correspondingAuthor":false,"prefix":"","firstName":"Xiaocong","middleName":"","lastName":"Chen","suffix":""}],"badges":[],"createdAt":"2024-09-23 23:41:11","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5140601/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5140601/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1038/s41467-025-58744-z","type":"published","date":"2025-04-17T04:00:00+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":80878013,"identity":"1a1165fc-cc1d-47e3-a890-b005d79eee75","added_by":"auto","created_at":"2025-04-18 07:07:35","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1122439,"visible":true,"origin":"","legend":"","description":"","filename":"01NguyenetalNatureMedicineSept232024SUBMIT.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5140601/v1_covered_5f27add7-69fe-41c6-b5b5-85b8da99c6b9.pdf"}],"financialInterests":"\u003cb\u003eYes\u003c/b\u003e there is potential Competing Interest.\nM Yao is employed as CEO by Univfy Inc. and is board director, shareholder and stock optionee of Univfy; she is inventor or co-inventor on Univfy's issued and pending patents and receives payment from patent licensor (Stanford University). ET Nguyen, T Swanson, X Chen are employed by and received stock options from Univfy Inc. M Retzloff performs paid consulting work as Nexplanon trainer for Organon and is Treasurer for the Society for Reproductive Technology (SART).","formattedTitle":"Predicting IVF live birth probabilities using machine learning, center-specific models: validation results and potential benefits over national registry-based models.","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"nature-portfolio","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"","title":"Nature Portfolio","twitterHandle":"","acdcEnabled":false,"dfaEnabled":false,"editorialSystem":"ejp","reportingPortfolio":"","inReviewEnabled":true,"inReviewRevisionsEnabled":false},"keywords":"live birth probability, IVF live birth prediction, artificial intelligence, machine learning, SART, fertility prognosis","lastPublishedDoi":"10.21203/rs.3.rs-5140601/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5140601/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"Ongoing improvement of pretreatment live birth prognostication for in vitro fertilization (IVF) is critical for informing fertility patients' treatment decisions, advocating for IVF coverage and supporting value-based IVF care. The US national registry Society for Assisted Reproductive Technology (SART) IVF live birth prediction (LBP) model (SART model) has been widely adopted for its prognostic support without external validation or utilization studies. We conducted a retrospective model validation study to compare the IVF LBP performance of machine learning, center-specific (MLCS) models versus the SART model in 6 unrelated US fertility centers using their respective center-specific test sets comprising an aggregate of 4,635 patients' first-IVF cycle data. Compared to the SART model, MLCS2 showed higher median Precision Recall AUC at 0.75 (IQR 0.73, 0.77) vs. 0.69 (IQR 0.68, 0.71), p\u003c0.05 and higher median F1 Score across LBP thresholds. Further, MLCS1 showed no evidence of data drift when validated using out-of-time test data from a later period. Reclassification analysis showed that MLCS2 models assigned more appropriate and higher IVF LBPs compared to the SART model, which underestimated patient prognoses (continuous net reclassification index: 18.3%, p\u003c0.0001). Overall, MLCS2 and SART models assigned 30% of patients to differential prognostic groups, with MLCS2 assigning 26% of patients to a higher LBP category compared to the SART model. Importantly, MLCS2 models identified 11% of patients to have LBP ≥ 75%, whereas the SART model detected none. This group had a live birth rate of 81%. We recommend testing a larger sample of fertility centers to further evaluate MLCS model benefits and limitations.","manuscriptTitle":"Predicting IVF live birth probabilities using machine learning, center-specific models: validation results and potential benefits over national registry-based models.","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-10-09 05:29:23","doi":"10.21203/rs.3.rs-5140601/v1","editorialEvents":[],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"nature-communications","isNatureJournal":true,"hasQc":false,"allowDirectSubmit":false,"externalIdentity":"NCOMMS","sideBox":"Learn more about [Nature Communications](http://www.nature.com/ncomms/)","snPcode":"","submissionUrl":"https://mts-ncomms.nature.com/","title":"Nature Communications","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"ejp","reportingPortfolio":"Nature Communications","inReviewEnabled":true,"inReviewRevisionsEnabled":false}}],"origin":"","ownerIdentity":"9e321983-a8de-4284-8aa8-e9d91be146bb","owner":[],"postedDate":"October 9th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":38199829,"name":"Health sciences/Health care/Prognosis/Pregnancy outcome"},{"id":38199830,"name":"Scientific community and society/Business and industry/Technology"},{"id":38199831,"name":"Health sciences/Medical research/Outcomes research"},{"id":38199832,"name":"Health sciences/Diseases/Reproductive disorders/Infertility"},{"id":38199833,"name":"Health sciences/Health care/Health policy"}],"tags":[],"updatedAt":"2025-04-18T07:07:24+00:00","versionOfRecord":{"articleIdentity":"rs-5140601","link":"https://doi.org/10.1038/s41467-025-58744-z","journal":{"identity":"nature-communications","isVorOnly":false,"title":"Nature Communications"},"publishedOn":"2025-04-17 04:00:00","publishedOnDateReadable":"April 17th, 2025"},"versionCreatedAt":"2024-10-09 05:29:23","video":"","vorDoi":"10.1038/s41467-025-58744-z","vorDoiUrl":"https://doi.org/10.1038/s41467-025-58744-z","workflowStages":[]},"version":"v1","identity":"rs-5140601","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5140601","identity":"rs-5140601","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}