AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study

AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study Kridipaka Sindhvananda, Kewalin Ruengwattanachot, Surachai Leksuwankun, and 5 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8511457/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Background Medical students often struggle with headache disorders, which are common but diagnostically challenging due to limited clinical exposure and feedback during early clinical training. This study evaluated the feasibility and effectiveness of an AI-powered simulated patient (AISP) chatbot in enhancing history-taking and clinical reasoning for three primary headache disorders—migraine, tension-type, and cluster headache—among medical students at a tertiary university hospital in Thailand. Methods We developed a simulated patient platform that provided automated, rubric-based feedback across three headache cases. Third-year medical students who had completed pre-clerkship requirements participated in a mixed-methods experimental study with a pre-post design. The study consisted of a 1-week self-study phase followed by a pre-test Objective Structured Clinical Examination (OSCE), a 1-week washout period, and a 1-week AISP practice phase followed by a post-test OSCE. Quantitative outcomes included changes in OSCE scores, Clinical Reasoning Indicator-History Taking (CRI-HT) scores, usability/satisfaction measured using the Chatbot Usability Questionnaire (CUQ). Focus group discussions (FGDs) explored participants’ learning experiences, perceived system limitations, and recommendations for future implementation. Qualitative data were analyzed thematically. Results Twenty-five medical students participated. OSCE scores increased by 22.5 points ( p < 0.001, Cohen’s d = 1.93), and CRI-HT scores increased by 8.8 points ( p < 0.001, Cohen’s d = 1.81). CUQ findings indicated high usability. Students commonly cited the platform’s intuitive interface and informative feedback, although some noted the chatbot’s responses were overly robotic. FGDs further highlighted three themes: (1) convenient, structured practice support learning; (2) dialogue felt realistic but was limited for practicing communication skills; and (3) students recommended greater case variety and integration into the curriculum. Conclusions An AISP platform with automated, rubric-based feedback was feasible and associated with improvements in headache history-taking and clinical reasoning. It serves as a valuable complement to traditional teaching by enabling structured, repeated practice with timely feedback. AI Simulated Patient history-taking medical education headache clinical reasoning chatbot large language model OSCE Full Text Additional Declarations No competing interests reported. Supplementary Files Supplementary1.docx Supplementary2.docx Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 16 Mar, 2026 Reviews received at journal 12 Mar, 2026 Reviewers agreed at journal 12 Mar, 2026 Reviewers agreed at journal 05 Mar, 2026 Reviews received at journal 06 Feb, 2026 Reviewers agreed at journal 05 Feb, 2026 Reviewers invited by journal 05 Feb, 2026 Editor assigned by journal 08 Jan, 2026 Submission checks completed at journal 08 Jan, 2026 First submitted to journal 04 Jan, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8511457","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":586882178,"identity":"2c12f723-5219-4442-838c-3d8e5d9059df","order_by":0,"name":"Kridipaka Sindhvananda","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Kridipaka","middleName":"","lastName":"Sindhvananda","suffix":""},{"id":586882179,"identity":"ccfe1b77-fd80-4023-85ab-9fa5def6aa92","order_by":1,"name":"Kewalin Ruengwattanachot","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Kewalin","middleName":"","lastName":"Ruengwattanachot","suffix":""},{"id":586882180,"identity":"fedf78a1-bff2-4971-93a1-ca6ed0d8455e","order_by":2,"name":"Surachai Leksuwankun","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Surachai","middleName":"","lastName":"Leksuwankun","suffix":""},{"id":586882181,"identity":"06ab2260-bc80-4f6a-82b9-ac345719ce97","order_by":3,"name":"Thanakit Pongpitakmetha","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Thanakit","middleName":"","lastName":"Pongpitakmetha","suffix":""},{"id":586882182,"identity":"34011989-0343-4c1c-9b1a-d9b69c42b4d9","order_by":4,"name":"Akarin Hiransuthikul","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Akarin","middleName":"","lastName":"Hiransuthikul","suffix":""},{"id":586882183,"identity":"399f3192-8aed-4bd7-8645-7706429047b8","order_by":5,"name":"Totsapol Surawattanawong","email":"","orcid":"","institution":"King Chulalongkorn Memorial Hospital","correspondingAuthor":false,"prefix":"","firstName":"Totsapol","middleName":"","lastName":"Surawattanawong","suffix":""},{"id":586882184,"identity":"3165f5f5-ac16-4c30-a75a-a6b88f86baa6","order_by":6,"name":"Prakit Anukoolwittaya","email":"","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":false,"prefix":"","firstName":"Prakit","middleName":"","lastName":"Anukoolwittaya","suffix":""},{"id":586882185,"identity":"253be2ef-218a-43ab-b2cd-8e32d02e5877","order_by":7,"name":"Poonnakarn Panjasriprakarn","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7klEQVRIiWNgGAWjYPCCBAj1geEAAx+qEAEtjDOAWtggIgbEaWHmIUaLOXvzsccFNWny5rMb2B7b1NxJbGM/wPjhB8OfPFxaLHuOpRvPOJZjOOfOAXbjnGPPEtt4EpglexgMinFpMbiRYybNw1bBOEMigU06t+FwYhvQldJAicQGvFr+VdiDtViCtPA/YP5NUAtvW04iWAsjSAuIgU8L0C9p0rx9ackzJBLbDXuOHTZuk3jYZtljYIxTCyjEpHm+JdvOkEg+9uBHzWHZfv7kwzd+VMjhdhiCydgGYzSgiOPRAovEUTAKRsEoGAVoAAA0tVL2aoGqWAAAAABJRU5ErkJggg==","orcid":"","institution":"Chulalongkorn University","correspondingAuthor":true,"prefix":"","firstName":"Poonnakarn","middleName":"","lastName":"Panjasriprakarn","suffix":""}],"badges":[],"createdAt":"2026-01-04 08:23:15","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8511457/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8511457/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":102297687,"identity":"9990f8e4-ecb9-4387-9a0d-d0818093d1ed","added_by":"auto","created_at":"2026-02-10 10:28:45","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":745185,"visible":true,"origin":"","legend":"","description":"","filename":"AISPManuscriptFINAL.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8511457/v1_covered_fad90872-cf86-4ce8-9b2a-eac802471f0f.pdf"},{"id":102251630,"identity":"92fac2d1-7abd-4408-a7c5-5d56fe84bd05","added_by":"auto","created_at":"2026-02-09 19:50:38","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":972893,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementary1.docx","url":"https://assets-eu.researchsquare.com/files/rs-8511457/v1/7b598ff3ce5fd7e5497e144a.docx"},{"id":102251628,"identity":"24f7e416-5459-40cf-baa1-a78b3ce9be94","added_by":"auto","created_at":"2026-02-09 19:50:38","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":28128,"visible":true,"origin":"","legend":"","description":"","filename":"Supplementary2.docx","url":"https://assets-eu.researchsquare.com/files/rs-8511457/v1/19a18baa18c67d061b3cf35e.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study","fulltext":[],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":false,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":true,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":true,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"AI Simulated Patient, history-taking, medical education, headache, clinical reasoning, chatbot, large language model, OSCE","lastPublishedDoi":"10.21203/rs.3.rs-8511457/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8511457/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eMedical students often struggle with headache disorders, which are common but diagnostically challenging due to limited clinical exposure and feedback during early clinical training. This study evaluated the feasibility and effectiveness of an AI-powered simulated patient (AISP) chatbot in enhancing history-taking and clinical reasoning for three primary headache disorders\u0026mdash;migraine, tension-type, and cluster headache\u0026mdash;among medical students at a tertiary university hospital in Thailand.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eWe developed a simulated patient platform that provided automated, rubric-based feedback across three headache cases. Third-year medical students who had completed pre-clerkship requirements participated in a mixed-methods experimental study with a pre-post design. The study consisted of a 1-week self-study phase followed by a pre-test Objective Structured Clinical Examination (OSCE), a 1-week washout period, and a 1-week AISP practice phase followed by a post-test OSCE. Quantitative outcomes included changes in OSCE scores, Clinical Reasoning Indicator-History Taking (CRI-HT) scores, usability/satisfaction measured using the Chatbot Usability Questionnaire (CUQ). Focus group discussions (FGDs) explored participants\u0026rsquo; learning experiences, perceived system limitations, and recommendations for future implementation. Qualitative data were analyzed thematically.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eTwenty-five medical students participated. OSCE scores increased by 22.5 points (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001, Cohen\u0026rsquo;s \u003cem\u003ed\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.93), and CRI-HT scores increased by 8.8 points (\u003cem\u003ep\u003c/em\u003e\u0026thinsp;\u0026lt;\u0026thinsp;0.001, Cohen\u0026rsquo;s \u003cem\u003ed\u003c/em\u003e\u0026thinsp;=\u0026thinsp;1.81). CUQ findings indicated high usability. Students commonly cited the platform\u0026rsquo;s intuitive interface and informative feedback, although some noted the chatbot\u0026rsquo;s responses were overly robotic. FGDs further highlighted three themes: (1) convenient, structured practice support learning; (2) dialogue felt realistic but was limited for practicing communication skills; and (3) students recommended greater case variety and integration into the curriculum.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eAn AISP platform with automated, rubric-based feedback was feasible and associated with improvements in headache history-taking and clinical reasoning. It serves as a valuable complement to traditional teaching by enabling structured, repeated practice with timely feedback.\u003c/p\u003e","manuscriptTitle":"AI-Powered Simulated Patients with Automated Feedback for Enhancing Headache History-Taking Skills: A Convergent Mixed-Methods Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-02-09 19:50:33","doi":"10.21203/rs.3.rs-8511457/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-03-16T05:22:03+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-03-12T14:39:52+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"64701182989457370542361470795160453245","date":"2026-03-12T13:56:01+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"99921956882436148193375875369310315889","date":"2026-03-05T12:05:22+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-06T10:04:32+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"11519251668745568876367829167874844631","date":"2026-02-05T10:15:56+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-02-05T09:47:59+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-01-08T08:23:55+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-01-08T08:23:06+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2026-01-04T08:12:03+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"d6c8aee3-c895-4487-8dba-885531bad931","owner":[],"postedDate":"February 9th, 2026","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2026-05-19T11:25:03+00:00","versionOfRecord":[],"versionCreatedAt":"2026-02-09 19:50:33","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8511457","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8511457","identity":"rs-8511457","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}