AI Chatbot–Facilitated Clinical Simulation and Peer Role-Play in OSCE Preparation: A Pilot Randomized Controlled Trial on Feasibility and Educational Impact

AI Chatbot–Facilitated Clinical Simulation and Peer Role-Play in OSCE Preparation: A Pilot Randomized Controlled Trial on Feasibility and Educational Impact | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article AI Chatbot–Facilitated Clinical Simulation and Peer Role-Play in OSCE Preparation: A Pilot Randomized Controlled Trial on Feasibility and Educational Impact Hye-Yoon Lee, Jundong Kim, Hyojae Choi, Hyojin Bae, Aram Jeong, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7382682/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 24 Nov, 2025 Read the published version in BMC Medical Education → Version 1 posted 21 You are reading this latest preprint version Abstract Background: Artificial intelligence (AI) is increasingly applied in medical education, but its role in fostering interactive clinical competencies remains underexplored. This pilot study aimed to compare the feasibility and educational impact of an AI chatbot–based simulation with traditional peer role-play (PRP) for Objective Structured Clinical Examination (OSCE) preparation, and to share practical lessons from implementing a novel AI tool in a trial setting. Methods: Nineteen final-year Korean medicine students were randomly assigned to either an AI chatbot group (n = 9) or a PRP group (n = 10) after a baseline knowledge test. Both groups underwent a 30-minute physical examination practice session, followed by a one-hour clinical interview training session specific to their group. The AI chatbot group practiced with a text-based chatbot providing scenario-driven responses and automated feedback, while the PRP group practiced in pairs under tutor supervision. All participants then completed two OSCE stations (dizziness and shoulder pain). Performance was assessed using a structured checklist covering four domains: history taking, physical examination, patient education, and physician-patient interaction. Post-study questionnaires evaluated the learning experience. Results: Although the differences in OSCE scores between the groups did not reach statistical significance, several interesting and complementary trends were observed. For example, the PRP group tended to score higher in history taking (mean 74.4 vs. 66.2 in dizziness scenario, mean 54.5 vs 58.6 in shoulder pain scenario), while the AI chatbot group showed a tendency towards higher scores in patient education (32.5 vs. 22.2 in dizziness scenario, 85.0 vs. 66.7 in shoulder pain scenario). Survey results reflected these following trends. The PRP group valued the authenticity of the interaction and the exam-like environment. In contrast, the AI chatbot group reported higher satisfaction with the autonomy, opportunity for repetitive practice, and structured feedback. Conclusion: In this pilot study, AI chatbot–based training and PRP demonstrated complementary strengths for OSCE preparation. While PRP appears effective for developing performance-based procedural and communication skills in a realistic setting, AI chatbots show potential for fostering clinical reasoning in a self-paced, reflective learning environment. A blended learning model that strategically combines both methods may be the most effective approach to enhance students' overall clinical competence. Further research is needed to validate these preliminary findings. Artificial intelligence Large Language Model Chatbot Medical education Objective Structured Clinical Examination Peer role-play Clinical reasoning Simulation-based training Full Text Additional Declarations No competing interests reported. Supplementary Files Supplementary1.docx Supplementary2OSCELearningmethodsurvey.docx Supplementary3OSCEChatbotUsabilitySurvey.docx Cite Share Download PDF Status: Published Journal Publication published 24 Nov, 2025 Read the published version in BMC Medical Education → Version 1 posted Editorial decision: Revision requested 22 Sep, 2025 Reviews received at journal 21 Sep, 2025 Reviews received at journal 17 Sep, 2025 Reviews received at journal 14 Sep, 2025 Reviewers agreed at journal 12 Sep, 2025 Reviewers agreed at journal 12 Sep, 2025 Reviews received at journal 12 Sep, 2025 Reviewers agreed at journal 11 Sep, 2025 Reviewers agreed at journal 11 Sep, 2025 Reviewers agreed at journal 11 Sep, 2025 Reviews received at journal 11 Sep, 2025 Reviewers agreed at journal 11 Sep, 2025 Reviews received at journal 10 Sep, 2025 Reviewers agreed at journal 09 Sep, 2025 Reviewers agreed at journal 09 Sep, 2025 Reviewers agreed at journal 09 Sep, 2025 Reviewers invited by journal 09 Sep, 2025 Editor invited by journal 01 Sep, 2025 Editor assigned by journal 29 Aug, 2025 Submission checks completed at journal 29 Aug, 2025 First submitted to journal 15 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7382682","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":515286131,"identity":"45c3d19e-2e27-40e6-be5a-f98cd2182ac2","order_by":0,"name":"Hye-Yoon Lee","email":"","orcid":"","institution":"Pusan National University","correspondingAuthor":false,"prefix":"","firstName":"Hye-Yoon","middleName":"","lastName":"Lee","suffix":""},{"id":515286132,"identity":"ed7fcfc2-6218-4b10-a4d3-37d99103719c","order_by":1,"name":"Jundong Kim","email":"","orcid":"","institution":"Gachon University","correspondingAuthor":false,"prefix":"","firstName":"Jundong","middleName":"","lastName":"Kim","suffix":""},{"id":515286133,"identity":"345b4167-c18d-45ca-9d30-e1fe7271e51e","order_by":2,"name":"Hyojae Choi","email":"","orcid":"","institution":"Pusan National University","correspondingAuthor":false,"prefix":"","firstName":"Hyojae","middleName":"","lastName":"Choi","suffix":""},{"id":515286134,"identity":"5f4ba4de-b029-4b02-beb3-f4169795923c","order_by":3,"name":"Hyojin Bae","email":"","orcid":"","institution":"Seoul National University Medical Research Center","correspondingAuthor":false,"prefix":"","firstName":"Hyojin","middleName":"","lastName":"Bae","suffix":""},{"id":515286135,"identity":"23707b68-298c-4bf5-9937-e0cd8752e637","order_by":4,"name":"Aram Jeong","email":"","orcid":"","institution":"Gachon University","correspondingAuthor":false,"prefix":"","firstName":"Aram","middleName":"","lastName":"Jeong","suffix":""},{"id":515286136,"identity":"e24b904b-39a4-43e2-92f0-7b5e2c300486","order_by":5,"name":"Sungyoul Choi","email":"","orcid":"","institution":"Gachon University","correspondingAuthor":false,"prefix":"","firstName":"Sungyoul","middleName":"","lastName":"Choi","suffix":""},{"id":515286137,"identity":"bcdfa61b-6689-42e1-9d95-9b26421aef0b","order_by":6,"name":"Ji-Hwan Kim","email":"","orcid":"","institution":"Pusan National University","correspondingAuthor":false,"prefix":"","firstName":"Ji-Hwan","middleName":"","lastName":"Kim","suffix":""},{"id":515286138,"identity":"b25794b5-7370-4695-a404-c4315af5634a","order_by":7,"name":"Chang-Eop Kim","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA50lEQVRIiWNgGAWjYJACZhDBD2Mg0QS0SDaQrMXgALFa+Nl7D34uqLljt/n84oefCyruMPC3H2A2rsCjRbLnXLL0jGPPkrfdeGYsPePMMwaJMwnMiWfwaDG4kWPGzMN2ONnsxgEzZt62wwwMNxiYDzbg0WIP1vLvcLLxjOPfwFrkCWkxkMgBG25nwN8DscUAqCURnxaJM0C/8PYdTpC4wVMszXPmMI/hmcRmQ3xa+NuBIcbz7bA9f//xjZ95Kg7LyR0/fFgSnxYGBh4wmdggkQDjMuLXANNiz8B/gIDCUTAKRsEoGLEAAFepTCEqWj+SAAAAAElFTkSuQmCC","orcid":"","institution":"Gachon University","correspondingAuthor":true,"prefix":"","firstName":"Chang-Eop","middleName":"","lastName":"Kim","suffix":""}],"badges":[],"createdAt":"2025-08-15 15:53:17","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7382682/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7382682/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12909-025-08308-y","type":"published","date":"2025-11-24T15:57:39+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":97178389,"identity":"db24675d-0142-44e6-bfd2-c2b5721bf72b","added_by":"auto","created_at":"2025-12-01 16:09:30","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1167225,"visible":true,"origin":"","legend":"","description":"","filename":"20250824manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7382682/v1_covered_01d86619-9896-4a3d-abf6-0206c351877b.pdf"},{"id":91557252,"identity":"123d6a81-1b3f-46f3-877a-812ec0b53653","added_by":"auto","created_at":"2025-09-17 17:45:48","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":22467,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementary1.docx","url":"https://assets-eu.researchsquare.com/files/rs-7382682/v1/6cd0cdd8e403bd83a0a0dfc6.docx"},{"id":91557246,"identity":"7df68bc7-3fbc-4f43-ba70-46916b1ea63c","added_by":"auto","created_at":"2025-09-17 17:45:48","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":31167,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementary2OSCELearningmethodsurvey.docx","url":"https://assets-eu.researchsquare.com/files/rs-7382682/v1/ecbd8e0402b07f54eaac03c9.docx"},{"id":91557245,"identity":"185479ed-f2bd-4694-b474-32f879d09719","added_by":"auto","created_at":"2025-09-17 17:45:48","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":31665,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementary3OSCEChatbotUsabilitySurvey.docx","url":"https://assets-eu.researchsquare.com/files/rs-7382682/v1/ed93c613b8913d29ae9abe28.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"AI Chatbot–Facilitated Clinical Simulation and Peer Role-Play in OSCE Preparation: A Pilot Randomized Controlled Trial on Feasibility and Educational Impact","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Artificial intelligence, Large Language Model, Chatbot, Medical education, Objective Structured Clinical Examination, Peer role-play, Clinical reasoning, Simulation-based training","lastPublishedDoi":"10.21203/rs.3.rs-7382682/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7382682/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cb\u003eBackground:\u003c/b\u003e\u003c/p\u003e\u003cp\u003eArtificial intelligence (AI) is increasingly applied in medical education, but its role in fostering interactive clinical competencies remains underexplored. This pilot study aimed to compare the feasibility and educational impact of an AI chatbot\u0026ndash;based simulation with traditional peer role-play (PRP) for Objective Structured Clinical Examination (OSCE) preparation, and to share practical lessons from implementing a novel AI tool in a trial setting.\u003c/p\u003e\u003cp\u003e\u003cb\u003eMethods:\u003c/b\u003e\u003c/p\u003e\u003cp\u003eNineteen final-year Korean medicine students were randomly assigned to either an AI chatbot group (n\u0026thinsp;=\u0026thinsp;9) or a PRP group (n\u0026thinsp;=\u0026thinsp;10) after a baseline knowledge test. Both groups underwent a 30-minute physical examination practice session, followed by a one-hour clinical interview training session specific to their group. The AI chatbot group practiced with a text-based chatbot providing scenario-driven responses and automated feedback, while the PRP group practiced in pairs under tutor supervision. All participants then completed two OSCE stations (dizziness and shoulder pain). Performance was assessed using a structured checklist covering four domains: history taking, physical examination, patient education, and physician-patient interaction. Post-study questionnaires evaluated the learning experience.\u003c/p\u003e\u003cp\u003e\u003cb\u003eResults:\u003c/b\u003e\u003c/p\u003e\u003cp\u003eAlthough the differences in OSCE scores between the groups did not reach statistical significance, several interesting and complementary trends were observed. For example, the PRP group tended to score higher in history taking (mean 74.4 vs. 66.2 in dizziness scenario, mean 54.5 vs 58.6 in shoulder pain scenario), while the AI chatbot group showed a tendency towards higher scores in patient education (32.5 vs. 22.2 in dizziness scenario, 85.0 vs. 66.7 in shoulder pain scenario). Survey results reflected these following trends. The PRP group valued the authenticity of the interaction and the exam-like environment. In contrast, the AI chatbot group reported higher satisfaction with the autonomy, opportunity for repetitive practice, and structured feedback.\u003c/p\u003e\u003cp\u003e\u003cb\u003eConclusion:\u003c/b\u003e\u003c/p\u003e\u003cp\u003eIn this pilot study, AI chatbot\u0026ndash;based training and PRP demonstrated complementary strengths for OSCE preparation. While PRP appears effective for developing performance-based procedural and communication skills in a realistic setting, AI chatbots show potential for fostering clinical reasoning in a self-paced, reflective learning environment. A blended learning model that strategically combines both methods may be the most effective approach to enhance students' overall clinical competence. Further research is needed to validate these preliminary findings.\u003c/p\u003e","manuscriptTitle":"AI Chatbot–Facilitated Clinical Simulation and Peer Role-Play in OSCE Preparation: A Pilot Randomized Controlled Trial on Feasibility and Educational Impact","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-09-17 17:45:43","doi":"10.21203/rs.3.rs-7382682/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2025-09-22T06:48:30+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-21T16:21:28+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-18T00:49:23+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-14T05:16:22+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"152535449717981294790641617990483200845","date":"2025-09-12T07:38:57+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"262901682965629588969248325290001213123","date":"2025-09-12T06:55:31+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-12T05:34:45+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"313897214830154202908246937707274856728","date":"2025-09-12T02:47:38+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"159792925609543042848490277323290893238","date":"2025-09-12T02:31:02+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"272475882976014755085858001156951627168","date":"2025-09-11T21:14:45+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-11T08:30:32+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"47868395221570342679178036791791759215","date":"2025-09-11T08:14:56+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-09-10T15:58:47+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"105609589130969925814507448358149206561","date":"2025-09-10T00:57:16+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"22052062836583513407650868673248949279","date":"2025-09-09T20:59:27+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"156721601243603661498550043701056311866","date":"2025-09-09T20:34:26+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-09-09T20:29:53+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-09-01T11:21:45+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-08-29T11:43:52+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-08-29T11:41:56+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2025-08-15T15:45:13+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"fba79c96-17df-4fab-a7c0-786e7efae67c","owner":[],"postedDate":"September 17th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-12-01T16:01:51+00:00","versionOfRecord":{"articleIdentity":"rs-7382682","link":"https://doi.org/10.1186/s12909-025-08308-y","journal":{"identity":"bmc-medical-education","isVorOnly":false,"title":"BMC Medical Education"},"publishedOn":"2025-11-24 15:57:39","publishedOnDateReadable":"November 24th, 2025"},"versionCreatedAt":"2025-09-17 17:45:43","video":"","vorDoi":"10.1186/s12909-025-08308-y","vorDoiUrl":"https://doi.org/10.1186/s12909-025-08308-y","workflowStages":[]},"version":"v1","identity":"rs-7382682","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7382682","identity":"rs-7382682","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}