Performance of Large Language Models in Nursing Licensure Examinations: A Systematic Review and Meta-Analysis

Performance of Large Language Models in Nursing Licensure Examinations: A Systematic Review and Meta-Analysis | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Systematic Review Performance of Large Language Models in Nursing Licensure Examinations: A Systematic Review and Meta-Analysis Isaac Amankwaa; PhD, MSN, RN, Alex Odoom, Adams Kasim, Emmanuel Kobiah, and 4 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8671859/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Objectives : This systematic review and meta-analysis assessed the performance of large language models (LLMS) in nursing licensure examinations. Despite the increasing use of LLMS in healthcare education, their capabilities in nursing licensure examinations remain uncertain. This study provides evidence on the accuracy and limitations of LLMs to help guide their integration into nursing education and licensure. Design : The systematic review and meta-analysis adhered to PRISMA 2020 guidelines. Data sources : PubMed, CINAHL, PsycINFO, EMCARE, and ERIC were searched from April to June 2025. Eligibility criteria: Studies were eligible if they evaluated LLMs (e.g., GPT-4, ChatGPT, Qwen-2.5) using multiple-choice nursing licensure questions under exam-like conditions and reported quantitative accuracy. Open-ended items were excluded from the meta-analysis but narratively synthesised Review methods : Two reviewers independently screened, extracted data, and appraised the risk of bias. A random-effects meta-analysis estimated pooled accuracy; subgroup and meta-regression analyses explored heterogeneity. Results : Twelve studies assessed 13,870 MCQs across five exam systems and eight LLMs. Pooled accuracy was 69.6% (95% CI: 65.6–73.6%) with substantial heterogeneity (I² = 98%). GPT-4 outperformed GPT-3.5 (77.2% vs. 60.4%, six studies); domain-customised and newer models reached 93.6%. LLMs excelled in general medicine and pharmacology but underperformed in ethics and psychosocial integrity. Accuracy differed significantly by exam system (p < 0.01), but not by question difficulty (p = 0.90) or format (p = 0.96). Translated NCLEX-RN items reduced accuracy (p = 0.03); CNNLE was the only system with a significant positive effect (p < 0.001). Methodological variability and underreporting of model parameters were common. Conclusions: LLMs show promise for low-stakes educational applications, such as formative assessments within hybrid teaching models; however, they are unsuitable for unmoderated, high-stakes licensure decisions due to inconsistent performance. Regulatory guidelines, equitable access, and nursing-specific model development are needed to ensure fairness and validity. Research must prioritise standardised frameworks, error analysis, and broader geographic representation to address these limitations. Nursing Artificial Intelligence Large Language Models Licensure Nursing Educational Measurement Systematic Review Nursing Full Text Additional Declarations The authors declare no competing interests. Supplementary Files 1.SupplementaryFileS1aPRISMA2020checklist.docx 1a.SupplementaryFileS1bPRISMA2020abstractchecklist.docx 2.SupplementaryfileS2Dataextractiontemplate.docx 3.SupplementaryFileS4StandardisedClassification.docx 4.SupplementaryFileS3SummaryofStudiesExcluded.docx 5.SupplementaryFileS5QualityAssessment.docx 6.SupplementaryFileS8Funnelplots.docx 7.SupplementaryFileS6Metaregression.docx 8.SupplementaryFileS7Sensitivityanalysis.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8671859","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Systematic Review","associatedPublications":[],"authors":[{"id":578885169,"identity":"db367b99-bdc3-4c9b-a38e-896236f0ce18","order_by":0,"name":"Isaac Amankwaa; PhD, MSN, RN","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABA0lEQVRIiWNgGAWjYBAC+QY4kw1MyhHUwobONGZgYCZRS2IDQS38hx9++MFgF80/uy1N4keNXXr/jPwDDD9qGPKQ3IyqRSLNWLKHITl3xp1jxyR7jgEZN5IZGHuOMRQz4tTCYCANdH5uw430ths8bCBGMgMDbwNDYjNOhx3//JuBoT53PlDLzT//6tPlQbb8BWppw+n9HDOgLYdzN9xIO3abt+1wggFQCzPIlh5cWiRyyix7DI7nbryRlv5btu+44cYzjw0OyxyTSJyBQ4t8//HNN35UVOfOu5FmbPjmW7W83PHEhw/f1NgkzsfhfQgwQOMfYGCQwKd+FIyCUTAKRgEBAADOolhQyLo3iAAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0002-7879-4731","institution":"School of Clinical Sciences, Auckland University of Technology, Auckland, New Zealand","correspondingAuthor":true,"prefix":"","firstName":"MS","middleName":"Isaac Amankwaa;","lastName":"PhD","suffix":"PhD"},{"id":578885170,"identity":"d9189805-4505-4875-b392-d26d1b87ddce","order_by":1,"name":"Alex Odoom","email":"","orcid":"","institution":"2.\tDepartment of Medical Microbiology, University of Ghana Medical School, P. O. Box KB 4236, Korle Bu, Accra, Ghana.","correspondingAuthor":false,"prefix":"","firstName":"Alex","middleName":"","lastName":"Odoom","suffix":""},{"id":578885171,"identity":"3be2dcc2-55b9-4028-9147-ed3e606722c2","order_by":2,"name":"Adams Kasim","email":"","orcid":"","institution":"3.\tNursing and Midwifery Training College, Techiman, Krobo, Ghana","correspondingAuthor":false,"prefix":"","firstName":"Adams","middleName":"","lastName":"Kasim","suffix":""},{"id":578885172,"identity":"f19504c2-c6aa-4c99-b5bf-6d8b762b6624","order_by":3,"name":"Emmanuel Kobiah","email":"","orcid":"","institution":"4.\tDepartment of Nursing, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana","correspondingAuthor":false,"prefix":"","firstName":"Emmanuel","middleName":"","lastName":"Kobiah","suffix":""},{"id":578885173,"identity":"26ac73cc-46a0-403a-892d-079020d13296","order_by":4,"name":"Maximous Diebieri","email":"","orcid":"","institution":"5.\tNursing and Midwifery Training College, Kpembe, Salaga, Ghana","correspondingAuthor":false,"prefix":"","firstName":"Maximous","middleName":"","lastName":"Diebieri","suffix":""},{"id":578885174,"identity":"c2e2b8fa-5673-48c5-98e1-fa41fe4f44c2","order_by":5,"name":"Edward Appiah Boateng","email":"","orcid":"","institution":"4.\tDepartment of Nursing, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana","correspondingAuthor":false,"prefix":"","firstName":"Edward","middleName":"Appiah","lastName":"Boateng","suffix":""},{"id":578885175,"identity":"e5122f81-21ce-4152-b542-145a2bce9c1e","order_by":6,"name":"Sebastian Gyamfi","email":"","orcid":"","institution":"6.\tFaculty of Nursing, University of Windsor, Windsor, Ontario, Canada","correspondingAuthor":false,"prefix":"","firstName":"Sebastian","middleName":"","lastName":"Gyamfi","suffix":""},{"id":578885176,"identity":"2845952c-84d0-4cc4-925c-60055b7b163a","order_by":7,"name":"Caz Hales","email":"","orcid":"","institution":"7.\tFaculty of Health, School of Nursing, Midwifery, and Health Practice, Victoria University of Wellington, Wellington, New Zealand","correspondingAuthor":false,"prefix":"","firstName":"Caz","middleName":"","lastName":"Hales","suffix":""}],"badges":[],"createdAt":"2026-01-22 16:25:47","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-8671859/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8671859/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":101204494,"identity":"81b2d378-8d1d-41d0-b0ef-daed9d154d58","added_by":"auto","created_at":"2026-01-27 09:43:21","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":398913,"visible":true,"origin":"","legend":"","description":"","filename":"3.Manuscript.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/49b2acd9ac1d0b78b7d9f448.docx"},{"id":101203509,"identity":"646d050a-7148-4330-9b19-946e7b4a3143","added_by":"auto","created_at":"2026-01-27 09:39:56","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342,"visible":true,"origin":"","legend":"","description":"","filename":"rs8671859.json","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/32956a8d447d6fad68c17a6f.json"},{"id":101203971,"identity":"88016c55-4be3-4f35-9408-b6bef38f1818","added_by":"auto","created_at":"2026-01-27 09:41:11","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":184026,"visible":true,"origin":"","legend":"","description":"","filename":"rs86718590enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/b306781638298d446b373f9a.xml"},{"id":101204589,"identity":"ec01e6cf-a01c-4ccd-96ce-4d146d97c436","added_by":"auto","created_at":"2026-01-27 09:43:33","extension":"eps","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":741162,"visible":true,"origin":"","legend":"","description":"","filename":"drawingimage1.eps","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/991020d5ad9575a8e569cb73.eps"},{"id":101203586,"identity":"bfa3f464-12d0-403a-92f9-363e17e74298","added_by":"auto","created_at":"2026-01-27 09:40:07","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":25651,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/e74358ca139d8ff4d2f08fe4.png"},{"id":101204499,"identity":"4e302585-9166-4c11-a36f-02764c12e2dd","added_by":"auto","created_at":"2026-01-27 09:43:22","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21600,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/a6f0147a49f8bc24a9c93cc5.png"},{"id":101204313,"identity":"16557335-714f-4a93-bacd-14460dc52d2d","added_by":"auto","created_at":"2026-01-27 09:42:38","extension":"jpeg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":683223,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/5a76e61245ccf488225fa3da.jpeg"},{"id":101203866,"identity":"ac86e39f-7101-4d2c-897d-e2f7fdb115fc","added_by":"auto","created_at":"2026-01-27 09:40:49","extension":"jpeg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1074,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/1c7958ce66788fa3a4f77153.jpeg"},{"id":101204942,"identity":"9807ffc3-3421-4616-a72f-57ee65d59cd9","added_by":"auto","created_at":"2026-01-27 09:45:06","extension":"jpeg","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1117483,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage4.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/5e76e6a9fb97ce0787a19858.jpeg"},{"id":101204713,"identity":"1ac1f2fe-2b07-4a6b-a04a-1bb74d2d77ae","added_by":"auto","created_at":"2026-01-27 09:43:47","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":24341,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/a09a7f7f95b32bb455e1e228.png"},{"id":101203752,"identity":"47be42bd-bd47-4cd8-946f-85ccaf2b4207","added_by":"auto","created_at":"2026-01-27 09:40:36","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":22814,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/889fd2500e9e6fbe956031c0.png"},{"id":101203973,"identity":"9ca741f0-e167-45ab-b01e-fc8e370578ee","added_by":"auto","created_at":"2026-01-27 09:41:11","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23693,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/c9bec7c6fd0bcb849e91de74.png"},{"id":101751241,"identity":"32076242-8eff-4a1c-82f3-5dd65a695e8f","added_by":"auto","created_at":"2026-02-03 10:18:40","extension":"png","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":22143,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/a751733b16a38362157b41cc.png"},{"id":101203617,"identity":"3d3becb8-404b-4d2c-8f6d-977493c40c44","added_by":"auto","created_at":"2026-01-27 09:40:13","extension":"png","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":15227,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/9f6ae1d79c143d8e427dc552.png"},{"id":101203212,"identity":"88ea5ec9-7deb-4a38-b49c-1cea7f23f2bc","added_by":"auto","created_at":"2026-01-27 09:39:05","extension":"png","order_by":14,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":9575,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/748038dc6f78f62e8ca395e7.png"},{"id":101942734,"identity":"6f9ea9ac-4b35-44ae-aa06-3c698c6b15df","added_by":"auto","created_at":"2026-02-05 09:36:09","extension":"png","order_by":15,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21082,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage10.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/48c84c79a206b255b9e5d415.png"},{"id":101204853,"identity":"f7bf6130-0d35-4780-b7ff-e1fe749f41ce","added_by":"auto","created_at":"2026-01-27 09:44:48","extension":"png","order_by":16,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":129846,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/b1918bd988f86d0a3a35f7cd.png"},{"id":101003382,"identity":"3e9273f0-33e1-4f4a-bcae-c6240a7bad37","added_by":"auto","created_at":"2026-01-23 17:04:16","extension":"png","order_by":17,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":935,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage3.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/f8e51165404e065764230436.png"},{"id":101203552,"identity":"5c9a5ecd-f090-4eb8-a841-bbcce302f4b9","added_by":"auto","created_at":"2026-01-27 09:40:02","extension":"png","order_by":18,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":276300,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/ceaa0fd185676a09d9d16629.png"},{"id":101003386,"identity":"591eaebd-304f-4077-98db-417830b18030","added_by":"auto","created_at":"2026-01-23 17:04:17","extension":"png","order_by":19,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23549,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/56ee5b3be14500273ada2df5.png"},{"id":101204841,"identity":"89493289-b62a-44e6-a876-273b90f2872a","added_by":"auto","created_at":"2026-01-27 09:44:11","extension":"png","order_by":20,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":22262,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage6.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/df337383c6f024c40278eb60.png"},{"id":101204053,"identity":"cef52751-c282-4036-9de3-c3e192f1296e","added_by":"auto","created_at":"2026-01-27 09:41:27","extension":"png","order_by":21,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":23063,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage7.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/64d5fc115ad5ef99bb0527a1.png"},{"id":101296730,"identity":"424f94e5-bbfd-4501-81fa-5131d595cb6f","added_by":"auto","created_at":"2026-01-28 09:19:32","extension":"png","order_by":22,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":21691,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage8.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/210c27fd777c6e38fbb14213.png"},{"id":101203251,"identity":"ab177a5c-2676-414c-b37b-11343875e914","added_by":"auto","created_at":"2026-01-27 09:39:12","extension":"png","order_by":23,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":14899,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage9.png","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/8c8b86ea1883bcf2718336f3.png"},{"id":101003387,"identity":"4fdae225-4f83-4f62-8640-a70cab3c521b","added_by":"auto","created_at":"2026-01-23 17:04:17","extension":"xml","order_by":24,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":181688,"visible":true,"origin":"","legend":"","description":"","filename":"rs86718590structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/ac6857c26da3279852437267.xml"},{"id":101203584,"identity":"3d980f18-5de6-409f-9645-310594dc94f6","added_by":"auto","created_at":"2026-01-27 09:40:07","extension":"html","order_by":25,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":196283,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/f3cf26d66f6b5764524b1a6f.html"},{"id":101943960,"identity":"4b82f1fa-331a-41a2-9b64-8fad843ff3d4","added_by":"auto","created_at":"2026-02-05 09:46:39","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":888397,"visible":true,"origin":"","legend":"","description":"","filename":"3.Manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1_covered_185f8e71-d25e-4cb6-8a26-9577c9fefa67.pdf"},{"id":101003359,"identity":"ca89e2f5-53f4-4a32-a2fa-3b55421458e0","added_by":"auto","created_at":"2026-01-23 17:04:14","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":33478,"visible":true,"origin":"","legend":"","description":"","filename":"1.SupplementaryFileS1aPRISMA2020checklist.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/3755e84974fd501a6b19a314.docx"},{"id":101204511,"identity":"b3665f5e-f160-421d-9eb6-a3f05227ad76","added_by":"auto","created_at":"2026-01-27 09:43:25","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":270795,"visible":true,"origin":"","legend":"","description":"","filename":"1a.SupplementaryFileS1bPRISMA2020abstractchecklist.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/9ab5a866834fe3f92520294b.docx"},{"id":101203571,"identity":"9dfddbcc-0749-47e5-bfc7-68719a2c4dae","added_by":"auto","created_at":"2026-01-27 09:40:05","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":17175,"visible":true,"origin":"","legend":"","description":"","filename":"2.SupplementaryfileS2Dataextractiontemplate.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/03f43a2866654cd66649fd52.docx"},{"id":101204994,"identity":"f0e1a5d8-8e48-44d4-93ea-7bad45ac7297","added_by":"auto","created_at":"2026-01-27 09:45:16","extension":"docx","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":18618,"visible":true,"origin":"","legend":"","description":"","filename":"3.SupplementaryFileS4StandardisedClassification.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/2a937f815f36663c5a8dddcc.docx"},{"id":101203713,"identity":"5fdaad36-793b-4d93-a084-9b1930f1402c","added_by":"auto","created_at":"2026-01-27 09:40:29","extension":"docx","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":17443,"visible":true,"origin":"","legend":"","description":"","filename":"4.SupplementaryFileS3SummaryofStudiesExcluded.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/fb0523af08acc382f2cbf702.docx"},{"id":101203553,"identity":"e236835f-29a9-4ebc-9465-c9748a397892","added_by":"auto","created_at":"2026-01-27 09:40:02","extension":"docx","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":32609,"visible":true,"origin":"","legend":"","description":"","filename":"5.SupplementaryFileS5QualityAssessment.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/4957e494903d4ca3dd73c8f5.docx"},{"id":101203985,"identity":"4dc7444f-3bfe-4e60-a839-27098e8ab02e","added_by":"auto","created_at":"2026-01-27 09:41:13","extension":"docx","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":50790,"visible":true,"origin":"","legend":"","description":"","filename":"6.SupplementaryFileS8Funnelplots.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/8e1ea0286b6820809bddb2a3.docx"},{"id":101203865,"identity":"975d0f9b-1c3e-4443-9fe7-98aae66ff1c7","added_by":"auto","created_at":"2026-01-27 09:40:48","extension":"docx","order_by":8,"title":"","display":"","copyAsset":false,"role":"supplement","size":23776,"visible":true,"origin":"","legend":"","description":"","filename":"7.SupplementaryFileS6Metaregression.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/ce97aef1cb1f15c1851aedd3.docx"},{"id":101204857,"identity":"eafce67f-2051-454e-8a91-f3de04cb0ac6","added_by":"auto","created_at":"2026-01-27 09:44:49","extension":"docx","order_by":9,"title":"","display":"","copyAsset":false,"role":"supplement","size":18113,"visible":true,"origin":"","legend":"","description":"","filename":"8.SupplementaryFileS7Sensitivityanalysis.docx","url":"https://assets-eu.researchsquare.com/files/rs-8671859/v1/ec5dcc6fe78c6ad8c7880642.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003ePerformance of Large Language Models in Nursing Licensure Examinations: A Systematic Review and Meta-Analysis\u003c/strong\u003e\u003c/p\u003e","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Auckland University of Technology","isAcceptedByJournal":false,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Artificial Intelligence, Large Language Models, Licensure, Nursing, Educational Measurement, Systematic Review, Nursing","lastPublishedDoi":"10.21203/rs.3.rs-8671859/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8671859/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eObjectives\u003c/strong\u003e: This systematic review and meta-analysis assessed the performance of large language models (LLMS) in nursing licensure examinations. Despite the increasing use of LLMS in healthcare education, their capabilities in nursing licensure examinations remain uncertain. This study provides evidence on the accuracy and limitations of LLMs to help guide their integration into nursing education and licensure.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDesign\u003c/strong\u003e: The systematic review and meta-analysis adhered to PRISMA 2020 guidelines.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData sources\u003c/strong\u003e: PubMed, CINAHL, PsycINFO, EMCARE, and ERIC were searched from April to June 2025.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEligibility criteria:\u003c/strong\u003e Studies were eligible if they evaluated LLMs (e.g., GPT-4, ChatGPT, Qwen-2.5) using multiple-choice nursing licensure questions under exam-like conditions and reported quantitative accuracy. Open-ended items were excluded from the meta-analysis but narratively synthesised\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eReview methods\u003c/strong\u003e: Two reviewers independently screened, extracted data, and appraised the risk of bias. A random-effects meta-analysis estimated pooled accuracy; subgroup and meta-regression analyses explored heterogeneity.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults\u003c/strong\u003e: Twelve studies assessed 13,870 MCQs across five exam systems and eight LLMs. Pooled accuracy was 69.6% (95% CI: 65.6–73.6%) with substantial heterogeneity (I² = 98%). GPT-4 outperformed GPT-3.5 (77.2% vs. 60.4%, six studies); domain-customised and newer models reached 93.6%. LLMs excelled in general medicine and pharmacology but underperformed in ethics and psychosocial integrity. Accuracy differed significantly by exam system (p \u0026lt; 0.01), but not by question difficulty (p = 0.90) or format (p = 0.96). Translated NCLEX-RN items reduced accuracy (p = 0.03); CNNLE was the only system with a significant positive effect (p \u0026lt; 0.001). Methodological variability and underreporting of model parameters were common.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions: \u003c/strong\u003eLLMs show promise for low-stakes educational applications, such as formative assessments within hybrid teaching models; however, they are unsuitable for unmoderated, high-stakes licensure decisions due to inconsistent performance. Regulatory guidelines, equitable access, and nursing-specific model development are needed to ensure fairness and validity. Research must prioritise standardised frameworks, error analysis, and broader geographic representation to address these limitations.\u003c/p\u003e","manuscriptTitle":"Performance of Large Language Models in Nursing Licensure Examinations: A Systematic Review and Meta-Analysis","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-01-23 17:04:09","doi":"10.21203/rs.3.rs-8671859/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1a57966e-21f5-4169-9c56-538215ee2da8","owner":[],"postedDate":"January 23rd, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":61594533,"name":"Nursing"}],"tags":[],"updatedAt":"2026-01-23T17:04:09+00:00","versionOfRecord":[],"versionCreatedAt":"2026-01-23 17:04:09","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8671859","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8671859","identity":"rs-8671859","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}