Teacher–Student–Machine Interaction Autonomous Learning: A Structured LLM-Integrated Framework for Developing Independent Clinical Reasoning in Residency Training

Teacher–Student–Machine Interaction Autonomous Learning: A Structured LLM-Integrated Framework for Developing Independent Clinical Reasoning in Residency Training | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Teacher–Student–Machine Interaction Autonomous Learning: A Structured LLM-Integrated Framework for Developing Independent Clinical Reasoning in Residency Training pengru Wang, He Li, Dingyuan Tu, Mengli Chang, Qiying Zhang, Gan Xu, and 3 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8818923/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 10 You are reading this latest preprint version Abstract Background This study evaluated whether a structured pedagogical framework integrating large language models (LLMs) into residency training could develop clinical reasoning competencies that transfer to independent performance. Methods In this prospective randomized controlled trial, residents were assigned to Teacher–Student–Machine Interaction Autonomous Learning (TSM-AL) group or traditional teaching group. The TSM-AL framework positioned LLMs as supervised cognitive auxiliaries within a five-step case analysis process featuring controlled information release, predefined reasoning tasks, and structured utilization guidelines. Six formative assessments were conducted over 16 weeks, followed by summative examinations under both LLM-assisted and unassisted conditions. Outcomes included clinical reasoning scores, entrustable professional activity (EPA) levels, global faculty ratings, expert-aligned diagnostic accuracy, and standardized patient examination performance. Results Significant between-group differences in diagnostic pathway completeness emerged at week 14 (3.53 ± 1.36 vs 2.68 ± 1.49, P = 0.018) and week 16 (3.35 ± 1.43 vs 2.37 ± 1.51, P = 0.009). In the unassisted final examination, the TSM-AL group demonstrated significantly higher clinical reasoning scores (7.32 ± 2.30 vs 5.12 ± 1.66, P < 0.001). EPA distributions differed significantly (P = 0.034), with fewer TSM-AL residents requiring complete supervision (11.76% vs 29.41%). Global faculty ratings also differed significantly (P = 0.019), with 23.53% of TSM-AL residents achieving Level 5 compared to none in the traditional group. Expert alignment for core problem identification (67.65% vs 44.12%, P = 0.042) and differential diagnosis matching (73.53% vs 50.00%, P = 0.046) favored the TSM-AL group. Standardized patient examination scores were significantly higher in the TSM-AL group (80.88 ± 3.82 vs 78.71 ± 3.90, P = 0.025). Notably, TSM-AL residents performed better without LLM assistance than with assistance. Conclusions The TSM-AL framework significantly enhanced clinical reasoning competencies that transferred to independent performance, demonstrating that structured LLM integration develops autonomous reasoning rather than fostering technological dependency. Clinical reasoning Competency-based medical education Residency training Large language models Artificial intelligence in medical education Entrustable professional activities Self-directed learning Figures Figure 1 Figure 2 Figure 3 Introduction The fundamental goal of clinical medical education is to cultivate physicians who are capable of independently and safely delivering patient care in real-world clinical settings. Achieving this goal requires mastery of clinical reasoning, as clinical practice is inherently a case-centered cognitive and decision-making process that involves the integration of history and physical examination findings, formulation of clinical problem representations, generation and prioritization of differential diagnoses, and execution of diagnostic and therapeutic decisions [ 1 , 2 ]. Clinical reasoning skills are therefore essential to effective clinical practice and play a critical role in ensuring diagnostic accuracy and evidence-based patient management [ 3 ]. However, a growing body of evidence consistently indicates that medical students and junior residents demonstrate substantial deficiencies in case analysis and diagnostic reasoning [ 4 , 5 ]. Deficits in clinical reasoning have been identified as the leading cause of diagnostic error, with potentially serious consequences for patient safety [ 6 ]. Accordingly, identifying effective approaches to developing clinical case analysis competency has become an urgent priority in contemporary medical education. Despite the recognized importance of clinical reasoning, current educational approaches remain insufficient to systematically cultivate this competency. Traditional lecture-based teaching approaches primarily emphasize knowledge transmission and are inherently limited in their capacity to explicitly present, model, and train complete clinical reasoning processes[ 7 ]. Although problem-based learning and case-based learning models have been widely promoted to support case-centered education, their implementation remains constrained by several practical challenges, including heavy reliance on faculty expertise, limited availability of high-quality case materials, inconsistent feedback, and difficulties in scaling instruction[ 8 – 11 ]. These limitations reflect not merely the shortcomings of individual teaching methods but a broader structural misalignment between educational objectives and the competencies required in authentic clinical workplaces [ 12 ]. Consequently, learners may perform adequately on theoretical examinations yet demonstrate insufficient case analysis and decision-making competence in real or simulated clinical contexts[ 12 , 13 ]. With the rapid advancement of generative artificial intelligence, large language models (LLMs) have increasingly been adopted in medical education and have begun to exert tangible influence on case-based teaching practices. Reported applications of LLMs include virtual patient simulation, interactive case-based dialogue, automated feedback generation, and clinical decision support [ 14 , 15 ]. These tools are often promoted for their potential to expand case exposure, deliver timely personalized feedback, and partially alleviate faculty resource constraints [ 16 , 17 ]. Despite this growing adoption, many current implementations rest on implicit assumptions about how learners engage with LLM-generated information, and these assumptions warrant careful examination. In practice, learners and educators frequently use LLMs as substitutes for evidence-based literature retrieval and clinical knowledge resources, bypassing essential processes of source verification and critical appraisal [ 18 ]. This concern is compounded by the phenomenon of AI hallucination, whereby LLMs may produce coherent but factually inaccurate outputs, including incorrect diagnostic criteria, misleading management suggestions, or fabricated references [ 19 , 20 ]. Beyond issues of factual reliability, many existing applications position learners as passive recipients of AI-generated content rather than active constructors of clinical reasoning, which may foster cognitive dependency and weaken the development of independent analytical skills [ 21 ]. Importantly, in most current educational uses, learners’ patterns of LLM engagement—how prompts are formulated, how outputs are interpreted, and how AI-derived information is integrated into clinical reasoning—are rarely subjected to structured faculty supervision or process-level evaluation. Consequently, it remains unclear whether AI-supported learning experiences meaningfully translate into sustainable, independent clinical competence once technological support is withdrawn [ 12 ]. Taken together, these limitations suggest that the educational value of LLMs depends not on their generative capacity alone, but on how they are pedagogically positioned, supervised, and assessed within clinical traini ng. In response, this study implements a competency-based educational reform integrating large language models into clinical case teaching through a structured instructional framework. The reform operationalizes LLM use within a supervised, stepwise case analysis process, with predefined reasoning tasks, controlled information release, and explicit documentation of AI interaction. Faculty oversight and competency-aligned assessment are embedded to ensure that learners’ reasoning processes and use of LLMs remain observable and evaluable. Method Study Design and Participant Allocation This prospective, randomized controlled educational intervention study was conducted at a single academic medical center between September 2024 and June 2025. Participants were residents enrolled in standardized orthopedic surgery residency training programs. Following baseline competency assessment, residents were randomly assigned in a 1:1 ratio to either the Teacher–Student–Machine Interaction Autonomous Learnin (TSM-AL) group or the traditional teaching group using a computer-generated randomization sequence. Allocation concealment was ensured through the use of sequentially numbered, opaque, sealed envelopes prepared by an investigator not involved in participant recruitment or instructional activities. To ensure internal validity, both groups received equivalent training conditions with respect to residency stage, core didactic content, clinical case themes, teaching faculty, total instructional hours, and assessment timepoints. The only difference between groups was the implementation of a structured large language model–assisted autonomous learning framework in the educational reform group. Conceptual Framework: Teacher–Student–Machine Interaction Autonomous Learning The educational reform was grounded in a pedagogical framework termed Teacher–Student–Machine Interaction Autonomous Learning , developed to support structured integration of artificial intelligence tools into medical education while preserving the primacy of human clinical reasoning (Fig 1). Within this framework, three interacting agents were assigned clearly defined roles. Faculty served as supervisors, facilitators, and evaluators, responsible for instructional design, monitoring reasoning processes, and ensuring appropriate use of artificial intelligence tools. Resident learners functioned as the primary reasoning agents and decision-makers, assuming responsibility for clinical problem solving while developing metacognitive awareness of their reasoning processes. The large language model was positioned as a supervised cognitive auxiliary rather than an authoritative diagnostic source, explicitly preventing its use as an “answer provider.” Instruction was organized into two complementary phases: synchronous, faculty-guided in-class instruction (Phase I), which allowed real-time observation and feedback, and asynchronous self-directed learning with remote supervision (Phase II), during which residents independently engaged with clinical problems while documenting their reasoning processes for subsequent faculty review. Case Design and Instructional Materials Clinical cases were derived from authentic patient encounters to enhance ecological validity. Case selection prioritized common clinical presentations with high reasoning value, clear opportunities for anatomical localization, and relevance to risk-stratified clinical decision-making. Case information was structured for staged release to simulate progressive clinical data acquisition. Initial case presentations deliberately excluded diagnostically directive information to discourage premature closure and promote systematic hypothesis generation. Five-Step Teaching Model The core pedagogical intervention consisted of a structured five-step teaching model designed to externalize implicit clinical reasoning processes into observable, teachable, and assessable behaviors (Fig 2). In Step One , faculty presented foundational clinical information, including the history of present illness, relevant past medical history, and general physical examination findings, while intentionally withholding specialty-specific examination findings, imaging results, and laboratory data. This controlled information release prompted residents to identify core clinical problems and formulate anatomically grounded hypotheses. In Step Two , residents analyzed symptom distribution in relation to potentially affected neural structures and anatomical compartments and specified which specialty examinations would provide the greatest diagnostic discrimination. Large language model use was restricted to concept clarification and logical verification, with explicit prohibition of diagnostic requests. In Step Three , residents generated no more than three diagnostic hypotheses, identified the most likely diagnosis with supporting rationale, and designated the highest-risk alternative requiring exclusion. This constraint emphasized prioritized reasoning rather than exhaustive but noncommittal differential listing. In Step Four , new clinical information was introduced sequentially, requiring residents to document how specialty examination findings and imaging results modified, strengthened, or refuted their initial hypotheses. Faculty specifically observed residents’ flexibility in revising initial impressions when presented with disconfirming evidence. In Step Five , residents synthesized their reasoning into actionable clinical decisions and engaged in structured reflection, identifying key reasoning nodes, recognizing potential cognitive biases, and evaluating the utility and limitations of large language model assistance. Large Language Model Utilization Guidelines and Supervision Structured utilization guidelines positioned the large language model as an auxiliary cognitive tool rather than a diagnostic authority. Permitted uses included clarification of medical concepts, verification of anatomical relationships, and exploration of diagnostic logic. Prohibited uses included requesting specific diagnoses, generating differential diagnosis lists, or soliciting treatment recommendations. Supervision was maintained through direct observation during synchronous instruction and review of interaction logs during asynchronous learning. Residents documented model queries, responses, and annotations regarding information integration, which informed individualized faculty feedback (Fig 3). Assessment Framework A multi-timepoint assessment framework was used to evaluate intervention effects across multiple competency domains. Assessments were conducted at baseline, at six interim timepoints during the intervention, and at the conclusion of training, consistent with longitudinal competency assessment approaches in medical education[22, 23]. Summative assessment differed between groups to evaluate both assisted and independent performance. The educational reform group completed two assessments with large language model access followed by one assessment without model access, enabling evaluation of internalized competency independent of technological support. The traditional teaching group completed two parallel summative assessments. Assessment instruments evaluated three primary domains: clinical reasoning and problem-solving ability through structured case analyses; autonomous learning and large language model utilization competency through interaction log analysis and reflection quality; and practice-readiness competencies through direct observation[24, 25]. Detailed scoring rubrics are provided in the supplementary appendix. Data Management and Ethical Considerations All study data were collected and analyzed in accordance with institutional data protection policies. Participant identifiers were replaced with study codes, and linkage files were maintained under restricted access. Residents provided written informed consent, acknowledging that participation was voluntary and would not affect residency evaluations. The study protocol was approved by the institutional ethics committee. The research involved no modifications to actual patient care. All reasoning exercises were conducted using standardized case materials derived from—but not directly connected to—ongoing patient encounters. Study performance data were maintained separately from official residency evaluations to avoid potential coercion. Results Study Population and Baseline Characteristics A total of 68 orthopedic surgery residents enrolled in standardized residency training programs were recruited and randomly assigned to either the TSM-AL group (n = 34) or the traditional teaching group (n = 34). The mean age was 28.18 ± 3.41 years in the TSM-AL group and 27.99 ± 3.78 years in the traditional group (P = 0.667). The TSM-AL group included 22 male residents, while the traditional group included 18 males. Distribution across postgraduate training years was comparable between groups. In the TSM-AL group, Postgraduate Year (PGY) -1, PGY-2, and PGY-3 residents accounted for 11 (32.35%), 12 (35.29%), and 11 (32.35%) participants, respectively. Corresponding proportions in the traditional group were 10 (29.41%), 15 (44.12%), and 9 (26.47%), with no statistically significant difference between groups (P = 0.748). Educational background was similarly balanced. In the TSM-AL group, 14 residents (41.18%) held bachelor’s degrees, 7 (20.59%) held master’s degrees, and 13 (38.24%) held doctoral degrees. In the traditional group, 10 residents (29.41%) held bachelor’s degrees, 12 (35.29%) held master’s degrees, and 12 (35.29%) held doctoral degrees (P = 0.364). Baseline assessments demonstrated no significant differences between groups. Standardized patient examination scores were 71.56 ± 4.23 in the TSM-AL group and 71.15 ± 2.69 in the traditional group (P = 0.349). Baseline clinical reasoning scores were 3.47 ± 1.61 and 4.35 ± 2.14, respectively (P = 0.095). Distributions of entrustable professional activity (EPA) levels (P = 0.110) and global faculty ratings (P = 0.758) were also comparable between groups. Complete baseline characteristics and pre-training assessment results are presented in Table 1. Formative Assessment Outcomes During Training Six formative assessments were conducted at weeks 2, 4, 6, 12, 14, and 16. Diagnostic pathway completeness scores showed no significant between-group differences during the early training period. At week 2, scores were 2.00 ± 1.35 in the TSM-AL group and 2.03 ± 1.40 in the traditional group (P = 0.931). At week 4, scores were 2.18 ± 1.32 and 2.09 ± 1.50, respectively (P = 0.800). At week 6, scores were 2.29 ± 1.56 in the TSM-AL group and 1.94 ± 1.28 in the traditional group (P = 0.320). At week 12, the TSM-AL group demonstrated higher scores than the traditional group (3.26 ± 1.54 vs 2.68 ± 1.28), approaching statistical significance (P = 0.096). Statistically significant differences emerged at week 14 (3.53 ± 1.36 vs 2.68 ± 1.49, P = 0.018) and were maintained at week 16 (3.35 ± 1.43 vs 2.37 ± 1.51, P = 0.009). Within the TSM-AL group, scores reflecting appropriate use of large language models increased progressively over time, from 0.94 ± 0.87 at week 2 to 2.06 ± 0.97 at week 16, with the highest mean score observed at week 14 (2.26 ± 0.92). Formative assessment outcomes are summarized in Table 2. Final Assessment Outcomes Under LLM-Assisted Conditions Residents in the TSM-AL group completed a summative assessment under conditions permitting large language model assistance. The mean clinical reasoning score was 5.85 ± 1.99. EPA level distributions showed 6 residents (17.65%) at Level 1, 6 (17.65%) at Level 2, 7 (20.59%) at Level 3, 12 (35.29%) at Level 4, and 3 (8.82%) at Level 5. Global faculty rating distributions were 4 (11.77%) at Level 1, 6 (17.65%) at Level 2, 10 (29.41%) at Level 3, 8 (23.53%) at Level 4, and 6 (17.65%) at Level 5. The mean score for appropriate use of large language models was 4.82 ± 1.60. Expert-aligned diagnostic accuracy analysis showed that 21 residents (61.77%) correctly identified the core clinical problem consistent with expert consensus, 22 (64.71%) aligned with the primary diagnosis, 20 (58.82%) matched at least two of the top three differential diagnoses, and 21 (61.77%) proposed initial investigations concordant with expert recommendations. These outcomes are detailed in Table 3. Independent Performance in Non-LLM Final Examination Both groups completed a final summative assessment without access to large language models. The TSM-AL group achieved significantly higher clinical reasoning scores than the traditional group (7.32 ± 2.30 vs 5.12 ± 1.66, P < 0.001). EPA level distributions differed significantly between groups (P = 0.034). In the TSM-AL group, 3 residents (11.76%) were rated at Level 1, 7 (35.29%) at Level 2, 9 (26.47%) at Level 3, 8 (23.53%) at Level 4, and 1 (2.94%) at Level 5. In contrast, the traditional group showed 10 residents (29.41%) at Level 1, 8 (23.53%) at Level 2, 9 (8.82%) at Level 3, 4 (32.35%) at Level 4, and 2 (2.94%) at Level 5. Global faculty rating distributions also differed significantly between groups (P = 0.019). In the TSM-AL group, 3 residents (8.82%) were rated at Level 1, 6 (17.65%) at Level 2, 8 (23.53%) at Level 3, 9 (26.47%) at Level 4, and 8 (23.53%) at Level 5. In the traditional group, ratings were 9 (26.47%) at Level 1, 8 (23.53%) at Level 2, 10 (29.41%) at Level 3, 7 (20.59%) at Level 4, and none at Level 5. For expert-aligned diagnostic accuracy, alignment with expert consensus on the core clinical problem was achieved by 23 residents (67.65%) in the TSM-AL group and 15 residents (44.12%) in the traditional group (P = 0.042). Alignment with the primary diagnosis was achieved by 18 residents (53.94%) in the TSM-AL group and 16 residents (44.12%) in the traditional group (P = 0.804). Alignment on at least two of the top three diagnoses was achieved by 25 residents (73.53%) in the TSM-AL group and 17 residents (50.00%) in the traditional group (P = 0.046). Alignment on initial investigation selection was achieved by 16 residents (47.06%) in the TSM-AL group and 18 residents (52.94%) in the traditional group (P = 0.808). Standardized patient examination scores were significantly higher in the TSM-AL group than in the traditional group (80.88 ± 3.82 vs 78.71 ± 3.90, P = 0.025). Complete final assessment outcomes are presented in Table 4. Discussion This study examined whether a structured pedagogical framework integrating large language models (LLMs) into clinical case–based teaching could enhance orthopedic surgery residents’ independent clinical reasoning. Compared with traditional instruction, the Teacher–Student–Machine Interaction Autonomous Learning (TSM-AL) approach was associated with superior clinical reasoning performance, more favorable entrustable professional activity (EPA) distributions, and higher global faculty competency ratings. Crucially, these advantages persisted when residents were assessed without access to LLM support, suggesting that the observed gains reflected internalized and transferable reasoning skills rather than technology-dependent performance. LLMs into medical education has rapidly expanded across instructional domains, including virtual patient simulations, interactive case-based dialogue, and automated assessment generation [ 20 , 26 ]. Prior work has reported promising short-term outcomes; for example, Brügge and colleagues found that AI-simulated history taking with structured feedback improved clinical decision-making scores [ 18 ], and Wang and colleagues reported superior clinical examination performance using GPT-simulated patients compared with traditional role-playing [ 19 ]. However, much of this research has focused on immediate performance gains under supported conditions rather than on the development of independent clinical reasoning skills. Moreover, existing implementations commonly position learners as passive recipients of AI-generated content and lack structured faculty supervision to verify accuracy or guide reasoning processes[ 27 , 28 ], Systematic reviews have also highlighted the risks of hallucinated outputs and the propagation of incorrect clinical information [ 26 , 29 ]. Critically, few studies assess whether AI-assisted learning translates to sustained competency when assistance is withdrawn, leaving uncertainty about its impact on enduring, autonomous clinical competence [ 4 , 30 ]. Our results extend these findings by demonstrating that a structured pedagogical integration of LLMs, with explicit supervision and unassisted assessment, can support the internalization of clinical reasoning capabilities. In response to these limitations, the Teacher–Student–Machine Interaction Autonomous Learning framework established structured pedagogical boundaries that positioned large language models as cognitive scaffolds rather than substitutes for independent clinical reasoning. Central to this design was the deliberate sequencing of cognitive activities: residents were required to generate problem representations and initial diagnostic hypotheses before engaging with model outputs for verification and reflection. Under such constraints, model responses served as material for critical appraisal rather than definitive answers, reinforcing residents’ own reasoning processes. By requiring learners to articulate and commit to their reasoning prior to LLM interaction, the framework may have enhanced metacognitive awareness and reduced premature diagnostic closure, a common risk with unrestricted AI use. This structured engagement offers a plausible explanation for the persistence of superior performance in assessments conducted without model access, suggesting that the intervention promoted internalization of clinical reasoning processes rather than mere dependency on technological assistance. During the initial training period, diagnostic pathway completeness scores did not differ significantly between groups at weeks 2, 4, and 6, with the week 12 assessment approaching but not reaching statistical significance (P = 0.096), indicating a transitional phase. Statistically significant differences emerged by week 14 (3.53 ± 1.36 vs 2.68 ± 1.49, P = 0.018) and were sustained at week 16 (3.35 ± 1.43 vs 2.37 ± 1.51, P = 0.009). From a skill acquisition perspective, this trajectory aligns with deliberate practice and expertise development frameworks, which posit that complex cognitive skills require sustained engagement, iterative feedback, and refinement before measurable improvement is evident. The early phase likely reflected residents’ adaptation to the structured constraints of the five-step teaching model and the development of effective strategies for engaging with large language models. Scores for appropriate LLM use exhibited a parallel developmental pattern, increasing from 0.94 ± 0.87 at week 2 to 2.26 ± 0.92 at week 14, further indicating that strategic interaction with the models matured over time. In contrast to studies of simpler AI implementations, where performance improvements emerged immediately—for example, Wang and colleagues reported immediate advantages with GPT-simulated patients [ 19 ] and Hudon and colleagues observed higher concordance with AI-generated script concordance tests for trained models than untrained models (ρ = 0.64 vs ρ = 0.41) [ 31 ] —the present framework emphasized structured model utilization, critical appraisal of outputs, and integration with independent reasoning, necessitating extended practice to internalize effectively. These observations are consistent with educational theory emphasizing that clinical reasoning is a complex cognitive skill requiring deliberate practice, iterative feedback, varied case exposure, and explicit instruction in cognitive strategies [ 32 ], suggesting that the benefits of structured LLM engagement accrue progressively as learners master both problem solving and productive model interaction. The study employed a dual-context summative assessment framework to examine resident performance under both LLM-assisted and unassisted conditions, allowing evaluation of not only how residents engaged with AI support but also whether gains transferred to independent reasoning. Under LLM-assisted assessment, TSM-AL residents achieved a mean clinical reasoning score of 5.85 ± 1.99 and an appropriate large language model use score of 4.82 ± 1.60, indicating the development of disciplined and productive engagement strategies within the structured framework. Entrustable professional activity (EPA) distributions and global faculty competency ratings were similarly favorable, with over 40% of TSM-AL residents attaining Levels 4–5, and expert alignment rates ranged from 58.82% to 64.71% across core tasks including problem identification, primary diagnosis, differential diagnosis matching, and initial investigation planning. When assessed without LLM support, the TSM-AL group continued to outperform the traditional group on clinical reasoning (7.32 ± 2.30 vs 5.12 ± 1.66, P < 0.001), with a 43% relative advantage, and demonstrated significant differences in EPA distributions (P = 0.034) and faculty ratings (P = 0.019), including a notable proportion (23.53%) achieving Level 5 entrustment compared with none in the traditional group. Expert alignment analyses further showed significant benefits for core problem identification (67.65% vs 44.12%, P = 0.042) and differential diagnosis matching (73.53% vs 50.00%, P = 0.046). Together, these findings demonstrate that structured AI-assisted learning can support not only performance in the context of model assistance but also the internalization of clinical reasoning capabilities in unassisted settings, addressing concerns in the literature that short-term AI benefits may not translate into durable, independent competence [ 20 , 21 ], and responding to prior calls for rigorous evaluation of independent reasoning and transfer to real clinical environments [ 21 ]. Improvements observed in EPA levels and faculty ratings carry significance beyond their statistical magnitude, as these instruments reflect readiness for authentic clinical responsibility. Shifts in EPA distributions indicate meaningful changes in supervisory needs, with direct implications for patient care quality and training efficiency. The absence of Level 5 faculty ratings in the traditional group, compared with 23.53% in the TSM-AL group, underscores the perceived differences in organization, analytical depth, and presentation clarity among intervention residents. The convergence of findings across multiple assessment modalities—including clinical reasoning scores, EPA ratings, faculty evaluations, expert alignment metrics, and standardized patient examinations—strengthens confidence that the observed effects represent genuine enhancement of underlying reasoning capabilities rather than artifacts of a single assessment method. The higher standardized patient examination scores in the TSM-AL group (80.88 ± 3.82 vs 78.71 ± 3.90, P = 0.025), despite equivalent baseline performance, further support a differential educational impact. From a practical standpoint, these findings offer residency programs a viable pathway for enhancing clinical reasoning instruction within existing resource constraints. The framework leverages LLM capabilities to extend practice opportunities and provide immediate feedback while maintaining faculty oversight through structured supervision protocols—addressing common challenges in case availability, faculty time, and standardized patient access that the competency-based education literature has identified as barriers to effective clinical reasoning development. Multiple authors have warned that AI-assisted practice gains do not guarantee transfer to independent clinical reasoning ability, with controlled trials typically measuring immediate post-intervention outcomes while long-term independent competency remains uncertain. Fąferek and colleagues specifically identified the lack of in-depth assessment of independent reasoning and the absence of evaluation for transfer to real clinical environments as critical gaps in existing research [ 21 ]. From a practical perspective, these findings suggest that residency programs can leverage structured integration of large language models to enhance clinical reasoning instruction within existing resource constraints. By extending opportunities for case practice and providing timely feedback within defined supervision protocols, the framework helps address common challenges in competency-based training, such as limited case availability, faculty workload pressures, and restricted access to standardized patients. Nonetheless, several limitations warrant consideration. The single-center design within a single surgical specialty limits generalizability, as the reasoning demands and case profiles in orthopedic surgery may differ from those in other disciplines. Outcomes were assessed only at the conclusion of the training period, and the durability of the observed advantages into later stages of residency and independent clinical practice remains unknown. In addition, the rapid evolution of large language model capabilities introduces uncertainty regarding the stability of educational effects across future model architectures and interaction paradigms. Comparative studies examining different model types, prompting strategies, and interaction designs are currently limited and needed to inform best practices [ 33 ]. Conclusion This study shows that the educational impact of large language models in residency training depends on their integration within structured pedagogical design rather than on their generative capacity alone. When embedded in a deliberate, faculty-supervised clinical reasoning framework, the Teacher–Student–Machine Interaction Autonomous Learning model supported the internalization of independent and transferable clinical reasoning skills. The persistence of competency gains after withdrawal of model access suggests that strategic use of large language models can foster durable reasoning competence. These findings highlight the potential of intentional AI integration to advance competency-based surgical education while upholding professional judgment and clinical autonomy. Declarations Acknowledgments: We are grateful to the orthopedic surgery residents who participated in this study for their engagement and commitment. We also thank the faculty members and standardized patients at Changzheng Hospital for their contributions to teaching implementation, assessment, and data collection. No external funding was received for this study. Declaration of interest statement: The authors declare that they have no competing interests. The authors alone are responsible for the content and the writing of this article. Notes on contributors: Pengru Wang, Dingyuan Tu, He Li, and Yifei Li contributed equally. Pengru Wang led study design and supervision. Dingyuan Tu and He Li contributed to instructional design and analysis. Bo Li, Yingtian Wang, and Wei Xu provided senior academic oversight. Ethics approval and consent to participate: This study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Changzheng Hospital, Navy Military Medical University. Written informed consent was obtained from all participating orthopedic surgery residents, and participation was voluntary with no impact on training evaluation or career progression. Consent for publication : Not Applicable. Competing interests: The authors declare that they have no competing interests. The authors alone are responsible for the content and the writing of this article. Funding: This work was supported by the Doctor Assistant Program of the Navy Military Medicine University (SL30) and Education assistant Program (JXPY2023B08). Availability of data and materials: The datasets used and analysed during the current study are available from the corresponding author on reasonable request Author Contributions: Pengru Wang, Dingyuan Tu and He Li contributed equally to this study. Pengru Wang contributed to study conception and design, supervised the educational intervention, and led manuscript drafting. Dingyuan Tu contributed to curriculum design, data analysis, and manuscript preparation. He Li contributed to data acquisition, assessment implementation, and critical manuscript revision. Mengli Chang, Qiying Zhang, and Gan Xu supported participant coordination, data management, and formative assessment implementation. Bo Li, Yingtian Wang, and Wei Xu provided senior academic oversight, contributed to study design refinement, and critically reviewed the manuscript. All authors met the ICMJE criteria for authorship, approved the final version of the manuscript, and agree to be accountable for all aspects of the work. References Shrivastava SR, Bobhate PS, Makade J. Empowering undergraduate medical students with clinical reasoning skills. Journal of Clinical Sciences. 2025;22(3):199-201. Custers E. Training Clinical Reasoning: Historical and Theoretical Background. In: ten Cate O, Custers E, Durning SJ, eds. Principles and Practice of Case-based Clinical Reasoning Education: A Method for Preclinical Students. Cham (CH): Springer Copyright 2018, The Author(s). 2018. p. 21-33. Jay R, Davenport C, Patel R. Clinical reasoning-the essentials for teaching medical students, trainees and non-medical healthcare professionals. Br J Hosp Med (Lond). 2024;85(7):1-8. Delavari S, Barzkar F, R MJPR, et al. Teaching and learning clinical reasoning skill in undergraduate medical students: A scoping review. PLoS One. 2024;19(10):e0309606. Cahyaningrum YD, Suhoyo Y, Rahayu GR. Facilitating clinical reasoning for medical students in clinical settings: a scoping review. Korean J Med Educ. 2025;37(2):163-86. Plackett RL. Evaluation of an online learning tool to improve medical students' clinical reasoning skills: UCL (University College London); 2019. Zhu Y, Zhang J, Fei J, Fang H, Zhang Z. Problem-based learning and case-based learning in clinical practical teaching for gynecology residents: a narrative review. Advances in Medical Education and Practice. 2025:1269-79. Plackett R, Kassianos AP, Mylan S, Kambouri M, Raine R, Sheringham J. The effectiveness of using virtual patient educational tools to improve medical students’ clinical reasoning skills: a systematic review. BMC medical education. 2022;22(1):365. Wang J, Jiang Y, Fu X, et al. Evaluating the impact of interactive video-based case-based learning in clinical medical education: a randomized controlled trial. Frontiers in Medicine. 2025;12:1556018. Gasim MS, Ibrahim MH, Abushama WA, Hamed IM, Ali IA. Medical students’ perceptions towards implementing case-based learning in the clinical teaching and clerkship training. BMC Medical Education. 2024;24(1):200. Singh M, Collins L, Farrington R, et al. From principles to practice: embedding clinical reasoning as a longitudinal curriculum theme in a medical school programme. Diagnosis. 2022;9(2):184-94. Kämmer JE, Hautz WE, Krummrey G, et al. Effects of interacting with a large language model compared with a human coach on the clinical diagnostic process and outcomes among fourth-year medical students: study protocol for a prospective, randomised experiment using patient vignettes. BMJ Open. 2024;14(7):e087469. Cahyaningrum YD, Suhoyo Y, Rahayu GR. Facilitating clinical reasoning for medical students in clinical settings: a scoping review. Korean Journal of Medical Education. 2025;37(2):163. Potter L, Jefferies C. Enhancing communication and clinical reasoning in medical education: Building virtual patients with generative AI. Future healthcare journal. 2024;11:100043. Zhui L, Yhap N, Liping L, et al. Impact of large language models on medical education and teaching adaptations. JMIR Medical Informatics. 2024;12(1):e55933. Borg A, Jobs B, Huss V, et al. Enhancing clinical reasoning skills for medical students: a qualitative comparison of LLM-powered social robotic versus computer-based virtual patients within rheumatology. Rheumatology International. 2024;44(12):3041-51. Hui Z, Zewu Z, Jiao H, Yu C. Application of ChatGPT-assisted problem-based learning teaching method in clinical medical education. BMC Medical Education. 2025;25(1):1-7. Brügge E, Ricchizzi S, Arenbeck M, et al. Large language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ. 2024;24(1):1391. Wang Z, Fan TT, Li ML, Zhu NJ, Wang XC. Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. BMC Med Educ. 2025;25(1):1030. Skryd A, Lawrence K. ChatGPT as a Tool for Medical Education and Clinical Decision-Making on the Wards: Case Study. JMIR Form Res. 2024;8:e51346. Fąferek J, Kononowicz AA, Bogutska N, et al. Applying ChatGPT to plan and create a realistic collection of virtual patients for clinical reasoning training. BMC Med Educ. 2025;25(1):1277. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356(4):387-96. Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. Gummesson C, Alm S, Cederborg A, et al. Entrustable professional activities (EPAs) for undergraduate medical education - development and exploration of social validity. BMC Med Educ. 2023;23(1):635. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676-82. García-Torres D, Vicente Ripoll MA, Fernández Peris C, Mira Solves JJ. Enhancing Clinical Reasoning with Virtual Patients: A Hybrid Systematic Review Combining Human Reviewers and ChatGPT. Healthcare (Basel). 2024;12(22). Shrivastava S, Bobhate P, Makade J. Empowering undergraduate medical students with clinical reasoning skills. Journal of Clinical Sciences. 2025;22:199-201. Omiye J, Gui H, Rezaei S, Zou J, Daneshjou R. Large language models in medicine: the potentials and pitfalls2023. Montagna M, Chiabrando F, De Lorenzo R, Querini P. Clinical decision support systems during teaching: a hands-on comparison (Preprint). JMIR Medical Education. 2023;11. Plackett R, Kassianos AP, Mylan S, Kambouri M, Raine R, Sheringham J. The effectiveness of using virtual patient educational tools to improve medical students' clinical reasoning skills: a systematic review. BMC Med Educ. 2022;22(1):365. Hudon A, Phan V, Charlin B, Wittmer R. Teaching Clinical Reasoning in Health Care Professions Learners Using AI-Generated Script Concordance Tests: Mixed Methods Formative Evaluation. JMIR Form Res. 2025;9:e76618. Custers E. Training Clinical Reasoning: Historical and Theoretical Background. 2018. p. 21-33. Hui Z, Zewu Z, Jiao H, Yu C. Application of ChatGPT-assisted problem-based learning teaching method in clinical medical education. BMC Medical Education. 2025;25. Tables Table 1. Baseline Characteristics and Pre-training Assessment Scores Variable TSM-AL Group (n= 34) Traditional Group (n= 34) P value Age (years) 28.18 ± 3.41 27.99 ± 3.78 0.667 Gender (Male/Female) 22 (%) 18 (%) Training year PGY-1 11 (32.35) PGY-2 12 (35.29) PGY-3 11 (32.35) PGY-1 10 (29.41) PGY-2 15 (44.12) PGY-3 9 (26.47） 0.748 Degree Bachelor 14 (41.18) Master 7 (20.59) Doctor 13 (38.24) Bachelor 10 (29.41) Master 12 (35.29) Doctor 12 (35.29) 0.364 Standardized patient score 71.56 ± 4.23 71.15 ± 2.69 0.349 Clinical reasoning score (0–10) 3.47 ± 1.61 4.35 ± 2.14 0.095 EPA level (1–5) Level 1 4(11.76) Level 2 12(35.29) Level 3 9(26.47) Level 4 8(23.53) Level 5 1(2.94) Level 1 11(32.35) Level 2 8(23.53) Level 3 3(8.82) Level 4 11(32.35) Level 5 1(2.94) 0.110 Global faculty rating (1–5) Level 1 7(20.59) Level 2 9(26.47) Level 3 8(23.53) Level 4 9(26.47) Level 5 1(2.94) Level 1 5(14.71) Level 2 10(29.41) Level 3 7(20.59) Level 4 12(35.29) Level 5 0(0) 0.758 Table 2. Comparison of Formative Assessment Scores During Training Assessment Domain TSM-AL Group Traditional Group P value Formative assessment-1(2w) Diagnostic pathway completeness (0–6) 2.00 ± 1.35 2.03 ± 1.40 0.931 Appropriate use of LLM (0–4) 0.94 ± 0.87 — — Formative assessment-2(4w) Diagnostic pathway completeness (0–6) 2.18 ± 1.32 2.09 ± 1.50 0.800 Appropriate use of LLM (0–4) 1.12 ± 0.83 — — Formative assessment-3(6w) Diagnostic pathway completeness (0–6) 2.29 ± 1.56 1.94 ± 1.28 0.320 Appropriate use of LLM (0–4) 1.15 ± 0.91 — — Formative assessment-4(12w) Diagnostic pathway completeness (0–6) 3.26 ± 1.54 2.68 ± 1.28 0.096 Appropriate use of LLM (0–4) 1.41 ± 0.91 — — Formative assessment-5(14w) Diagnostic pathway completeness (0–6) 3.53± 1.36 2.68 ± 1.49 0.018 Appropriate use of LLM (0–4) 2.26 ± 0.92 — — Formative assessment-6(16w) Diagnostic pathway completeness (0–6) 3.35 ± 1.43 2.37 ± 1.51 0.009 Appropriate use of LLM (0–4) 2.06 ± 0.97 — — Table 3. Final Assessment Outcomes: LLM-assisted Examination Outcome TSM-AL Group Clinical reasoning score (0–10) 5.85 ± 1.99 EPA level (1–5) Level 1 6(17.65) Level 2 6(17.65) Level 3 7(20.59) Level 4 12(35.29) Level 5 3(8.82) Global faculty rating (1–5) Level 1 4(11.77) Level 2 6(17.65) Level 3 10(29.41) Level 4 8(23.53) Level 5 6(17.65) Appropriate use of LLM (0–8) 4.82 ± 1.60 Core problem aligned with expert 21(61.77%) Primary diagnosis aligned 22(64.71%) ≥2 of top 3 diagnoses aligned 20(58.82%) Initial investigation aligned 21(61.77%) Table 4. Final Assessment Outcomes: Non-LLM (Independent Performance) Examination Outcome TSM-AL Group Traditional Group P value Clinical reasoning score (0–10) 7.32 ± 2.30 5.12 ± 1.66 0.000 EPA level (1–5) Level 1 3(11.76) Level 2 7(35.29) Level 3 (26.47) Level 4 8(23.53) Level 5 1(2.94) Level 1 10(29.41) Level 2 8(23.53) Level 3 9(8.82) Level 4 4(32.35) Level 5 2(2.94) 0.034 Global faculty rating (1–5) Level 1 3(8.82) Level 2 6(17.65) Level 3 8(23.53) Level 4 9(26.47) Level 5 8(23.53) Level 1 9(26.47) Level 2 8(23.53) Level 3 10(29.41) Level 4 7(20.59) Level 5 0(0) 0.019 Core problem aligned with expert 23(67.65%) 15(44.12%) 0.042 Primary diagnosis aligned 18(53.94%) 16(44.12%) 0.804 ≥2 of top 3 diagnoses aligned 25(73.53%) 17(50.00%) 0.046 Initial investigation aligned 16(47.06%) 18(52.94%) 0.808 Standardized patient score 80.88 ± 3.82 78.71 ± 3.90 0.025 Additional Declarations No competing interests reported. Supplementary Files AppendixA.docx AppendixC.docx AppendixB.docx Appendix3.png Appendix2.png Appendix1.png CONSORT2010Checklist.docx Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 04 May, 2026 Reviews received at journal 30 Apr, 2026 Reviews received at journal 22 Apr, 2026 Reviewers agreed at journal 22 Apr, 2026 Reviewers agreed at journal 18 Apr, 2026 Reviewers invited by journal 16 Apr, 2026 Editor invited by journal 08 Apr, 2026 Editor assigned by journal 27 Feb, 2026 Submission checks completed at journal 25 Feb, 2026 First submitted to journal 25 Feb, 2026 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8818923","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":627875542,"identity":"072b77b3-3d84-4667-b834-9206b75d838c","order_by":0,"name":"pengru Wang","email":"","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":false,"prefix":"","firstName":"pengru","middleName":"","lastName":"Wang","suffix":""},{"id":627875543,"identity":"1ac825fb-b3ea-46b4-b273-3d7a0cc68a89","order_by":1,"name":"He Li","email":"","orcid":"","institution":"Naval Medical Center","correspondingAuthor":false,"prefix":"","firstName":"He","middleName":"","lastName":"Li","suffix":""},{"id":627875544,"identity":"064d5e68-d982-482e-9cac-173aa5f92bdf","order_by":2,"name":"Dingyuan Tu","email":"","orcid":"","institution":"961st Hospital of Joint Logistic Support Force of PLA","correspondingAuthor":false,"prefix":"","firstName":"Dingyuan","middleName":"","lastName":"Tu","suffix":""},{"id":627875545,"identity":"64445f60-7c2a-4fe4-96ca-671e42f0eff0","order_by":3,"name":"Mengli Chang","email":"","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":false,"prefix":"","firstName":"Mengli","middleName":"","lastName":"Chang","suffix":""},{"id":627875550,"identity":"4120fc72-dd90-47da-88c2-aee170ef2008","order_by":4,"name":"Qiying Zhang","email":"","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":false,"prefix":"","firstName":"Qiying","middleName":"","lastName":"Zhang","suffix":""},{"id":627875553,"identity":"0023e938-fd99-4eaf-8282-27efccd4c265","order_by":5,"name":"Gan Xu","email":"","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":false,"prefix":"","firstName":"Gan","middleName":"","lastName":"Xu","suffix":""},{"id":627875555,"identity":"d9c58d99-1fc9-4036-9cf3-acc1884b1287","order_by":6,"name":"Bo Li","email":"","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":false,"prefix":"","firstName":"Bo","middleName":"","lastName":"Li","suffix":""},{"id":627875557,"identity":"08d278b1-65a1-488c-a476-3cf5563c840b","order_by":7,"name":"Yingting Wang","email":"","orcid":"","institution":"Chinese PLA General Hospital","correspondingAuthor":false,"prefix":"","firstName":"Yingting","middleName":"","lastName":"Wang","suffix":""},{"id":627875558,"identity":"29fde918-1b54-4907-8b8e-a12e6c6dcc0e","order_by":8,"name":"Wei Xu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA1klEQVRIiWNgGAWjYLCCBCBmY2BgfJBQUUOaFmaDB2eOkWYZm+TDFmbCygxu5Bg+eMBQl8cn3XysIrGBjYG/vTsBrxbJGTnGBgkMh4vZZI6l3UjcIcMgcebsBrxa+CVyt0kkMBxIbJPIMbuReIaNwUAiF78WNonc7T8SGOqAWvK/FSS2MRPWArIFGGLMIFvYGIjSItnz/rME2C8SacYSCWeO8RD0i8HxtMSPP4AhJj8j+eHHHxU1cvztvfi1gAHjP3BsggEPYeVQkEBQxSgYBaNgFIxcAACPCkRoRD2oagAAAABJRU5ErkJggg==","orcid":"","institution":"Shanghai Changzheng Hospital","correspondingAuthor":true,"prefix":"","firstName":"Wei","middleName":"","lastName":"Xu","suffix":""}],"badges":[],"createdAt":"2026-02-08 03:38:28","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8818923/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8818923/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":107832841,"identity":"8c3f1277-6524-48a6-a0fd-f3531531ac6c","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1689032,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic Representation of the LLM-Integrated Blended Learning Reform Model\u003c/p\u003e","description":"","filename":"Fig1.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/379be097ead8ec866758a5eb.png"},{"id":107832843,"identity":"4dfedecf-20b8-4026-bace-39597d4ad422","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":1419292,"visible":true,"origin":"","legend":"\u003cp\u003eSchematic representation of the five-step structured teaching model. The model progresses sequentially, guiding residents from initial information processing to synthesized clinical decision-making. The upper layer depicts faculty-controlled information release; the middle layer illustrates required resident cognitive tasks; the bottom layer indicates restricted phases of Large Language Model (LLM) assistance, emphasizing its role as a supportive rather than diagnostic tool.\u003c/p\u003e","description":"","filename":"Fig2.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/9b0fdd58a989b31c88f8b118.png"},{"id":107870408,"identity":"bc721d5b-45b6-4b3a-b387-ddec49b3bcd3","added_by":"auto","created_at":"2026-04-27 07:39:36","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":1294035,"visible":true,"origin":"","legend":"\u003cp\u003eOperational framework for controlled LLM utilization and supervision. The central interaction between resident and LLM is governed by strict guidelines defining the LLM as an auxiliary cognitive tool. The framework distinguishes between permitted conceptual inquiries and prohibited diagnostic requests. Supervision is maintained through a dual mechanism: direct observation during synchronous instruction and review of interaction logs during asynchronous learning, culminating in individualized faculty feedback\u003c/p\u003e","description":"","filename":"Fig3.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/6413e1a50e14eb09087699c3.png"},{"id":107872167,"identity":"4a15e176-927f-4399-80e5-fb9e0826d629","added_by":"auto","created_at":"2026-04-27 07:55:49","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":4905193,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/7ecfff90-cdc8-4d9c-a4c3-db8761ad5f5b.pdf"},{"id":107870710,"identity":"2e8da419-68b5-4962-9f76-1e8a8a2ec184","added_by":"auto","created_at":"2026-04-27 07:40:28","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":18031,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixA.docx","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/47089492fbf5ae5d2f9151de.docx"},{"id":107832844,"identity":"58eb795b-ef34-4836-8871-6e3285f4aafc","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":20130,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixC.docx","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/09a41906fea46a826ea7e285.docx"},{"id":107832846,"identity":"673cde73-1c50-4224-9568-f9e099047e4e","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":18244,"visible":true,"origin":"","legend":"","description":"","filename":"AppendixB.docx","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/9ea6996d02fa874e33e084e1.docx"},{"id":107870314,"identity":"29195ea2-c2b3-40f1-8223-1a8c6bcb43d3","added_by":"auto","created_at":"2026-04-27 07:39:19","extension":"png","order_by":4,"title":"","display":"","copyAsset":false,"role":"supplement","size":262001,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix3.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/1f5a83a5333fd3a1947241da.png"},{"id":107832848,"identity":"c7734e3b-1b29-4ab0-b165-3b41fef06555","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"png","order_by":5,"title":"","display":"","copyAsset":false,"role":"supplement","size":300936,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix2.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/86104c92e6f1ccdafa3cb6d1.png"},{"id":107869945,"identity":"5d9c03f0-a3c7-4b6e-8e69-0be5ede5a9dc","added_by":"auto","created_at":"2026-04-27 07:38:31","extension":"png","order_by":6,"title":"","display":"","copyAsset":false,"role":"supplement","size":445443,"visible":true,"origin":"","legend":"","description":"","filename":"Appendix1.png","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/bc9a9ff5a7f3268d72056b83.png"},{"id":107832850,"identity":"e16d600e-7dee-4996-a34c-4f7b44ddacbd","added_by":"auto","created_at":"2026-04-26 15:37:21","extension":"docx","order_by":7,"title":"","display":"","copyAsset":false,"role":"supplement","size":17192,"visible":true,"origin":"","legend":"","description":"","filename":"CONSORT2010Checklist.docx","url":"https://assets-eu.researchsquare.com/files/rs-8818923/v1/07e6f41979296dffaf2444bf.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Teacher–Student–Machine Interaction Autonomous Learning: A Structured LLM-Integrated Framework for Developing Independent Clinical Reasoning in Residency Training","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe fundamental goal of clinical medical education is to cultivate physicians who are capable of independently and safely delivering patient care in real-world clinical settings. Achieving this goal requires mastery of clinical reasoning, as clinical practice is inherently a case-centered cognitive and decision-making process that involves the integration of history and physical examination findings, formulation of clinical problem representations, generation and prioritization of differential diagnoses, and execution of diagnostic and therapeutic decisions [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e]. Clinical reasoning skills are therefore essential to effective clinical practice and play a critical role in ensuring diagnostic accuracy and evidence-based patient management [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. However, a growing body of evidence consistently indicates that medical students and junior residents demonstrate substantial deficiencies in case analysis and diagnostic reasoning [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. Deficits in clinical reasoning have been identified as the leading cause of diagnostic error, with potentially serious consequences for patient safety [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Accordingly, identifying effective approaches to developing clinical case analysis competency has become an urgent priority in contemporary medical education.\u003c/p\u003e \u003cp\u003eDespite the recognized importance of clinical reasoning, current educational approaches remain insufficient to systematically cultivate this competency. Traditional lecture-based teaching approaches primarily emphasize knowledge transmission and are inherently limited in their capacity to explicitly present, model, and train complete clinical reasoning processes[\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. Although problem-based learning and case-based learning models have been widely promoted to support case-centered education, their implementation remains constrained by several practical challenges, including heavy reliance on faculty expertise, limited availability of high-quality case materials, inconsistent feedback, and difficulties in scaling instruction[\u003cspan additionalcitationids=\"CR9 CR10\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. These limitations reflect not merely the shortcomings of individual teaching methods but a broader structural misalignment between educational objectives and the competencies required in authentic clinical workplaces [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Consequently, learners may perform adequately on theoretical examinations yet demonstrate insufficient case analysis and decision-making competence in real or simulated clinical contexts[\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eWith the rapid advancement of generative artificial intelligence, large language models (LLMs) have increasingly been adopted in medical education and have begun to exert tangible influence on case-based teaching practices. Reported applications of LLMs include virtual patient simulation, interactive case-based dialogue, automated feedback generation, and clinical decision support [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. These tools are often promoted for their potential to expand case exposure, deliver timely personalized feedback, and partially alleviate faculty resource constraints [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e]. Despite this growing adoption, many current implementations rest on implicit assumptions about how learners engage with LLM-generated information, and these assumptions warrant careful examination. In practice, learners and educators frequently use LLMs as substitutes for evidence-based literature retrieval and clinical knowledge resources, bypassing essential processes of source verification and critical appraisal [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e]. This concern is compounded by the phenomenon of AI hallucination, whereby LLMs may produce coherent but factually inaccurate outputs, including incorrect diagnostic criteria, misleading management suggestions, or fabricated references [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. Beyond issues of factual reliability, many existing applications position learners as passive recipients of AI-generated content rather than active constructors of clinical reasoning, which may foster cognitive dependency and weaken the development of independent analytical skills [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. Importantly, in most current educational uses, learners\u0026rsquo; patterns of LLM engagement\u0026mdash;how prompts are formulated, how outputs are interpreted, and how AI-derived information is integrated into clinical reasoning\u0026mdash;are rarely subjected to structured faculty supervision or process-level evaluation. Consequently, it remains unclear whether AI-supported learning experiences meaningfully translate into sustainable, independent clinical competence once technological support is withdrawn [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Taken together, these limitations suggest that the educational value of LLMs depends not on their generative capacity alone, but on how they are pedagogically positioned, supervised, and assessed within clinical traini ng.\u003c/p\u003e \u003cp\u003eIn response, this study implements a competency-based educational reform integrating large language models into clinical case teaching through a structured instructional framework. The reform operationalizes LLM use within a supervised, stepwise case analysis process, with predefined reasoning tasks, controlled information release, and explicit documentation of AI interaction. Faculty oversight and competency-aligned assessment are embedded to ensure that learners\u0026rsquo; reasoning processes and use of LLMs remain observable and evaluable.\u003c/p\u003e"},{"header":"Method","content":"\u003cp\u003e\u003cstrong\u003eStudy Design and Participant Allocation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis prospective, randomized controlled educational intervention study was conducted at a single academic medical center between September 2024 and June 2025. Participants were residents enrolled in standardized orthopedic surgery residency training programs. Following baseline competency assessment, residents were randomly assigned in a 1:1 ratio to either the Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learnin (TSM-AL) group or the traditional teaching group using a computer-generated randomization sequence. Allocation concealment was ensured through the use of sequentially numbered, opaque, sealed envelopes prepared by an investigator not involved in participant recruitment or instructional activities.\u003c/p\u003e\n\u003cp\u003eTo ensure internal validity, both groups received equivalent training conditions with respect to residency stage, core didactic content, clinical case themes, teaching faculty, total instructional hours, and assessment timepoints. The only difference between groups was the implementation of a structured large language model\u0026ndash;assisted autonomous learning framework in the educational reform group.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConceptual Framework: Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe educational reform was grounded in a pedagogical framework termed \u003cem\u003eTeacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning\u003c/em\u003e, developed to support structured integration of artificial intelligence tools into medical education while preserving the primacy of human clinical reasoning (Fig 1).\u003c/p\u003e\n\u003cp\u003eWithin this framework, three interacting agents were assigned clearly defined roles. Faculty served as supervisors, facilitators, and evaluators, responsible for instructional design, monitoring reasoning processes, and ensuring appropriate use of artificial intelligence tools. Resident learners functioned as the primary reasoning agents and decision-makers, assuming responsibility for clinical problem solving while developing metacognitive awareness of their reasoning processes. The large language model was positioned as a supervised cognitive auxiliary rather than an authoritative diagnostic source, explicitly preventing its use as an \u0026ldquo;answer provider.\u0026rdquo;\u003c/p\u003e\n\u003cp\u003eInstruction was organized into two complementary phases: synchronous, faculty-guided in-class instruction (Phase I), which allowed real-time observation and feedback, and asynchronous self-directed learning with remote supervision (Phase II), during which residents independently engaged with clinical problems while documenting their reasoning processes for subsequent faculty review.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCase Design and Instructional Materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eClinical cases were derived from authentic patient encounters to enhance ecological validity. Case selection prioritized common clinical presentations with high reasoning value, clear opportunities for anatomical localization, and relevance to risk-stratified clinical decision-making. Case information was structured for staged release to simulate progressive clinical data acquisition. Initial case presentations deliberately excluded diagnostically directive information to discourage premature closure and promote systematic hypothesis generation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFive-Step Teaching Model\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe core pedagogical intervention consisted of a structured five-step teaching model designed to externalize implicit clinical reasoning processes into observable, teachable, and assessable behaviors (Fig 2).\u003c/p\u003e\n\u003cp\u003eIn \u003cstrong\u003eStep One\u003c/strong\u003e, faculty presented foundational clinical information, including the history of present illness, relevant past medical history, and general physical examination findings, while intentionally withholding specialty-specific examination findings, imaging results, and laboratory data. This controlled information release prompted residents to identify core clinical problems and formulate anatomically grounded hypotheses.\u003c/p\u003e\n\u003cp\u003eIn \u003cstrong\u003eStep Two\u003c/strong\u003e, residents analyzed symptom distribution in relation to potentially affected neural structures and anatomical compartments and specified which specialty examinations would provide the greatest diagnostic discrimination. Large language model use was restricted to concept clarification and logical verification, with explicit prohibition of diagnostic requests.\u003c/p\u003e\n\u003cp\u003eIn \u003cstrong\u003eStep Three\u003c/strong\u003e, residents generated no more than three diagnostic hypotheses, identified the most likely diagnosis with supporting rationale, and designated the highest-risk alternative requiring exclusion. This constraint emphasized prioritized reasoning rather than exhaustive but noncommittal differential listing.\u003c/p\u003e\n\u003cp\u003eIn \u003cstrong\u003eStep Four\u003c/strong\u003e, new clinical information was introduced sequentially, requiring residents to document how specialty examination findings and imaging results modified, strengthened, or refuted their initial hypotheses. Faculty specifically observed residents\u0026rsquo; flexibility in revising initial impressions when presented with disconfirming evidence.\u003c/p\u003e\n\u003cp\u003eIn \u003cstrong\u003eStep Five\u003c/strong\u003e, residents synthesized their reasoning into actionable clinical decisions and engaged in structured reflection, identifying key reasoning nodes, recognizing potential cognitive biases, and evaluating the utility and limitations of large language model assistance.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLarge Language Model Utilization Guidelines and Supervision\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eStructured utilization guidelines positioned the large language model as an auxiliary cognitive tool rather than a diagnostic authority. Permitted uses included clarification of medical concepts, verification of anatomical relationships, and exploration of diagnostic logic. Prohibited uses included requesting specific diagnoses, generating differential diagnosis lists, or soliciting treatment recommendations. Supervision was maintained through direct observation during synchronous instruction and review of interaction logs during asynchronous learning. Residents documented model queries, responses, and annotations regarding information integration, which informed individualized faculty feedback (Fig 3).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAssessment Framework\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA multi-timepoint assessment framework was used to evaluate intervention effects across multiple competency domains. Assessments were conducted at baseline, at six interim timepoints during the intervention, and at the conclusion of training, consistent with longitudinal competency assessment approaches in medical education[22, 23].\u003c/p\u003e\n\u003cp\u003eSummative assessment differed between groups to evaluate both assisted and independent performance. The educational reform group completed two assessments with large language model access followed by one assessment without model access, enabling evaluation of internalized competency independent of technological support. The traditional teaching group completed two parallel summative assessments.\u003c/p\u003e\n\u003cp\u003eAssessment instruments evaluated three primary domains: clinical reasoning and problem-solving ability through structured case analyses; autonomous learning and large language model utilization competency through interaction log analysis and reflection quality; and practice-readiness competencies through direct observation[24, 25]. Detailed scoring rubrics are provided in the supplementary appendix.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData Management and Ethical Considerations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll study data were collected and analyzed in accordance with institutional data protection policies. Participant identifiers were replaced with study codes, and linkage files were maintained under restricted access. Residents provided written informed consent, acknowledging that participation was voluntary and would not affect residency evaluations. The study protocol was approved by the institutional ethics committee.\u003c/p\u003e\n\u003cp\u003eThe research involved no modifications to actual patient care. All reasoning exercises were conducted using standardized case materials derived from\u0026mdash;but not directly connected to\u0026mdash;ongoing patient encounters. Study performance data were maintained separately from official residency evaluations to avoid potential coercion.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eStudy Population and Baseline Characteristics\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 68 orthopedic surgery residents enrolled in standardized residency training programs were recruited and randomly assigned to either the TSM-AL group (n = 34) or the traditional teaching group (n = 34). The mean age was 28.18 \u0026plusmn; 3.41 years in the TSM-AL group and 27.99 \u0026plusmn; 3.78 years in the traditional group (P = 0.667). The TSM-AL group included 22 male residents, while the traditional group included 18 males. Distribution across postgraduate training years was comparable between groups. In the TSM-AL group, Postgraduate Year (PGY) -1, PGY-2, and PGY-3 residents accounted for 11 (32.35%), 12 (35.29%), and 11 (32.35%) participants, respectively. Corresponding proportions in the traditional group were 10 (29.41%), 15 (44.12%), and 9 (26.47%), with no statistically significant difference between groups (P = 0.748). Educational background was similarly balanced. In the TSM-AL group, 14 residents (41.18%) held bachelor\u0026rsquo;s degrees, 7 (20.59%) held master\u0026rsquo;s degrees, and 13 (38.24%) held doctoral degrees. In the traditional group, 10 residents (29.41%) held bachelor\u0026rsquo;s degrees, 12 (35.29%) held master\u0026rsquo;s degrees, and 12 (35.29%) held doctoral degrees (P = 0.364).\u003c/p\u003e\n\u003cp\u003eBaseline assessments demonstrated no significant differences between groups. Standardized patient examination scores were 71.56 \u0026plusmn; 4.23 in the TSM-AL group and 71.15 \u0026plusmn; 2.69 in the traditional group (P = 0.349). Baseline clinical reasoning scores were 3.47 \u0026plusmn; 1.61 and 4.35 \u0026plusmn; 2.14, respectively (P = 0.095). Distributions of entrustable professional activity (EPA) levels (P = 0.110) and global faculty ratings (P = 0.758) were also comparable between groups. Complete baseline characteristics and pre-training assessment results are presented in Table 1.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFormative Assessment Outcomes During Training\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSix formative assessments were conducted at weeks 2, 4, 6, 12, 14, and 16. Diagnostic pathway completeness scores showed no significant between-group differences during the early training period. At week 2, scores were 2.00 \u0026plusmn; 1.35 in the TSM-AL group and 2.03 \u0026plusmn; 1.40 in the traditional group (P = 0.931). At week 4, scores were 2.18 \u0026plusmn; 1.32 and 2.09 \u0026plusmn; 1.50, respectively (P = 0.800). At week 6, scores were 2.29 \u0026plusmn; 1.56 in the TSM-AL group and 1.94 \u0026plusmn; 1.28 in the traditional group (P = 0.320).\u003c/p\u003e\n\u003cp\u003eAt week 12, the TSM-AL group demonstrated higher scores than the traditional group (3.26 \u0026plusmn; 1.54 vs 2.68 \u0026plusmn; 1.28), approaching statistical significance (P = 0.096). Statistically significant differences emerged at week 14 (3.53 \u0026plusmn; 1.36 vs 2.68 \u0026plusmn; 1.49, P = 0.018) and were maintained at week 16 (3.35 \u0026plusmn; 1.43 vs 2.37 \u0026plusmn; 1.51, P = 0.009).\u003c/p\u003e\n\u003cp\u003eWithin the TSM-AL group, scores reflecting appropriate use of large language models increased progressively over time, from 0.94 \u0026plusmn; 0.87 at week 2 to 2.06 \u0026plusmn; 0.97 at week 16, with the highest mean score observed at week 14 (2.26 \u0026plusmn; 0.92). Formative assessment outcomes are summarized in Table 2.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFinal Assessment Outcomes Under LLM-Assisted Conditions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eResidents in the TSM-AL group completed a summative assessment under conditions permitting large language model assistance. The mean clinical reasoning score was 5.85 \u0026plusmn; 1.99. EPA level distributions showed 6 residents (17.65%) at Level 1, 6 (17.65%) at Level 2, 7 (20.59%) at Level 3, 12 (35.29%) at Level 4, and 3 (8.82%) at Level 5. Global faculty rating distributions were 4 (11.77%) at Level 1, 6 (17.65%) at Level 2, 10 (29.41%) at Level 3, 8 (23.53%) at Level 4, and 6 (17.65%) at Level 5. The mean score for appropriate use of large language models was 4.82 \u0026plusmn; 1.60.\u003c/p\u003e\n\u003cp\u003eExpert-aligned diagnostic accuracy analysis showed that 21 residents (61.77%) correctly identified the core clinical problem consistent with expert consensus, 22 (64.71%) aligned with the primary diagnosis, 20 (58.82%) matched at least two of the top three differential diagnoses, and 21 (61.77%) proposed initial investigations concordant with expert recommendations. These outcomes are detailed in Table 3.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eIndependent Performance in Non-LLM Final Examination\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBoth groups completed a final summative assessment without access to large language models. The TSM-AL group achieved significantly higher clinical reasoning scores than the traditional group (7.32 \u0026plusmn; 2.30 vs 5.12 \u0026plusmn; 1.66, P \u0026lt; 0.001).\u003c/p\u003e\n\u003cp\u003eEPA level distributions differed significantly between groups (P = 0.034). In the TSM-AL group, 3 residents (11.76%) were rated at Level 1, 7 (35.29%) at Level 2, 9 (26.47%) at Level 3, 8 (23.53%) at Level 4, and 1 (2.94%) at Level 5. In contrast, the traditional group showed 10 residents (29.41%) at Level 1, 8 (23.53%) at Level 2, 9 (8.82%) at Level 3, 4 (32.35%) at Level 4, and 2 (2.94%) at Level 5.\u003c/p\u003e\n\u003cp\u003eGlobal faculty rating distributions also differed significantly between groups (P = 0.019). In the TSM-AL group, 3 residents (8.82%) were rated at Level 1, 6 (17.65%) at Level 2, 8 (23.53%) at Level 3, 9 (26.47%) at Level 4, and 8 (23.53%) at Level 5. In the traditional group, ratings were 9 (26.47%) at Level 1, 8 (23.53%) at Level 2, 10 (29.41%) at Level 3, 7 (20.59%) at Level 4, and none at Level 5.\u003c/p\u003e\n\u003cp\u003eFor expert-aligned diagnostic accuracy, alignment with expert consensus on the core clinical problem was achieved by 23 residents (67.65%) in the TSM-AL group and 15 residents (44.12%) in the traditional group (P = 0.042). Alignment with the primary diagnosis was achieved by 18 residents (53.94%) in the TSM-AL group and 16 residents (44.12%) in the traditional group (P = 0.804). Alignment on at least two of the top three diagnoses was achieved by 25 residents (73.53%) in the TSM-AL group and 17 residents (50.00%) in the traditional group (P = 0.046). Alignment on initial investigation selection was achieved by 16 residents (47.06%) in the TSM-AL group and 18 residents (52.94%) in the traditional group (P = 0.808).\u003c/p\u003e\n\u003cp\u003eStandardized patient examination scores were significantly higher in the TSM-AL group than in the traditional group (80.88 \u0026plusmn; 3.82 vs 78.71 \u0026plusmn; 3.90, P = 0.025). Complete final assessment outcomes are presented in Table 4.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study examined whether a structured pedagogical framework integrating large language models (LLMs) into clinical case\u0026ndash;based teaching could enhance orthopedic surgery residents\u0026rsquo; independent clinical reasoning. Compared with traditional instruction, the Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning (TSM-AL) approach was associated with superior clinical reasoning performance, more favorable entrustable professional activity (EPA) distributions, and higher global faculty competency ratings. Crucially, these advantages persisted when residents were assessed without access to LLM support, suggesting that the observed gains reflected internalized and transferable reasoning skills rather than technology-dependent performance.\u003c/p\u003e \u003cp\u003eLLMs into medical education has rapidly expanded across instructional domains, including virtual patient simulations, interactive case-based dialogue, and automated assessment generation [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e]. Prior work has reported promising short-term outcomes; for example, Br\u0026uuml;gge and colleagues found that AI-simulated history taking with structured feedback improved clinical decision-making scores [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e], and Wang and colleagues reported superior clinical examination performance using GPT-simulated patients compared with traditional role-playing [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. However, much of this research has focused on immediate performance gains under supported conditions rather than on the development of independent clinical reasoning skills. Moreover, existing implementations commonly position learners as passive recipients of AI-generated content and lack structured faculty supervision to verify accuracy or guide reasoning processes[\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e], Systematic reviews have also highlighted the risks of hallucinated outputs and the propagation of incorrect clinical information [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e]. Critically, few studies assess whether AI-assisted learning translates to sustained competency when assistance is withdrawn, leaving uncertainty about its impact on enduring, autonomous clinical competence [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e, \u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e]. Our results extend these findings by demonstrating that a structured pedagogical integration of LLMs, with explicit supervision and unassisted assessment, can support the internalization of clinical reasoning capabilities.\u003c/p\u003e \u003cp\u003eIn response to these limitations, the Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning framework established structured pedagogical boundaries that positioned large language models as cognitive scaffolds rather than substitutes for independent clinical reasoning. Central to this design was the deliberate sequencing of cognitive activities: residents were required to generate problem representations and initial diagnostic hypotheses before engaging with model outputs for verification and reflection. Under such constraints, model responses served as material for critical appraisal rather than definitive answers, reinforcing residents\u0026rsquo; own reasoning processes. By requiring learners to articulate and commit to their reasoning prior to LLM interaction, the framework may have enhanced metacognitive awareness and reduced premature diagnostic closure, a common risk with unrestricted AI use. This structured engagement offers a plausible explanation for the persistence of superior performance in assessments conducted without model access, suggesting that the intervention promoted internalization of clinical reasoning processes rather than mere dependency on technological assistance.\u003c/p\u003e \u003cp\u003eDuring the initial training period, diagnostic pathway completeness scores did not differ significantly between groups at weeks 2, 4, and 6, with the week 12 assessment approaching but not reaching statistical significance (P\u0026thinsp;=\u0026thinsp;0.096), indicating a transitional phase. Statistically significant differences emerged by week 14 (3.53\u0026thinsp;\u0026plusmn;\u0026thinsp;1.36 vs 2.68\u0026thinsp;\u0026plusmn;\u0026thinsp;1.49, P\u0026thinsp;=\u0026thinsp;0.018) and were sustained at week 16 (3.35\u0026thinsp;\u0026plusmn;\u0026thinsp;1.43 vs 2.37\u0026thinsp;\u0026plusmn;\u0026thinsp;1.51, P\u0026thinsp;=\u0026thinsp;0.009). From a skill acquisition perspective, this trajectory aligns with deliberate practice and expertise development frameworks, which posit that complex cognitive skills require sustained engagement, iterative feedback, and refinement before measurable improvement is evident. The early phase likely reflected residents\u0026rsquo; adaptation to the structured constraints of the five-step teaching model and the development of effective strategies for engaging with large language models. Scores for appropriate LLM use exhibited a parallel developmental pattern, increasing from 0.94\u0026thinsp;\u0026plusmn;\u0026thinsp;0.87 at week 2 to 2.26\u0026thinsp;\u0026plusmn;\u0026thinsp;0.92 at week 14, further indicating that strategic interaction with the models matured over time. In contrast to studies of simpler AI implementations, where performance improvements emerged immediately\u0026mdash;for example, Wang and colleagues reported immediate advantages with GPT-simulated patients [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] and Hudon and colleagues observed higher concordance with AI-generated script concordance tests for trained models than untrained models (ρ\u0026thinsp;=\u0026thinsp;0.64 vs ρ\u0026thinsp;=\u0026thinsp;0.41) [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e] \u0026mdash;the present framework emphasized structured model utilization, critical appraisal of outputs, and integration with independent reasoning, necessitating extended practice to internalize effectively. These observations are consistent with educational theory emphasizing that clinical reasoning is a complex cognitive skill requiring deliberate practice, iterative feedback, varied case exposure, and explicit instruction in cognitive strategies [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e], suggesting that the benefits of structured LLM engagement accrue progressively as learners master both problem solving and productive model interaction.\u003c/p\u003e \u003cp\u003eThe study employed a dual-context summative assessment framework to examine resident performance under both LLM-assisted and unassisted conditions, allowing evaluation of not only how residents engaged with AI support but also whether gains transferred to independent reasoning. Under LLM-assisted assessment, TSM-AL residents achieved a mean clinical reasoning score of 5.85\u0026thinsp;\u0026plusmn;\u0026thinsp;1.99 and an appropriate large language model use score of 4.82\u0026thinsp;\u0026plusmn;\u0026thinsp;1.60, indicating the development of disciplined and productive engagement strategies within the structured framework. Entrustable professional activity (EPA) distributions and global faculty competency ratings were similarly favorable, with over 40% of TSM-AL residents attaining Levels 4\u0026ndash;5, and expert alignment rates ranged from 58.82% to 64.71% across core tasks including problem identification, primary diagnosis, differential diagnosis matching, and initial investigation planning. When assessed without LLM support, the TSM-AL group continued to outperform the traditional group on clinical reasoning (7.32\u0026thinsp;\u0026plusmn;\u0026thinsp;2.30 vs 5.12\u0026thinsp;\u0026plusmn;\u0026thinsp;1.66, P\u0026thinsp;\u0026lt;\u0026thinsp;0.001), with a 43% relative advantage, and demonstrated significant differences in EPA distributions (P\u0026thinsp;=\u0026thinsp;0.034) and faculty ratings (P\u0026thinsp;=\u0026thinsp;0.019), including a notable proportion (23.53%) achieving Level 5 entrustment compared with none in the traditional group. Expert alignment analyses further showed significant benefits for core problem identification (67.65% vs 44.12%, P\u0026thinsp;=\u0026thinsp;0.042) and differential diagnosis matching (73.53% vs 50.00%, P\u0026thinsp;=\u0026thinsp;0.046). Together, these findings demonstrate that structured AI-assisted learning can support not only performance in the context of model assistance but also the internalization of clinical reasoning capabilities in unassisted settings, addressing concerns in the literature that short-term AI benefits may not translate into durable, independent competence [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e], and responding to prior calls for rigorous evaluation of independent reasoning and transfer to real clinical environments [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eImprovements observed in EPA levels and faculty ratings carry significance beyond their statistical magnitude, as these instruments reflect readiness for authentic clinical responsibility. Shifts in EPA distributions indicate meaningful changes in supervisory needs, with direct implications for patient care quality and training efficiency. The absence of Level 5 faculty ratings in the traditional group, compared with 23.53% in the TSM-AL group, underscores the perceived differences in organization, analytical depth, and presentation clarity among intervention residents. The convergence of findings across multiple assessment modalities\u0026mdash;including clinical reasoning scores, EPA ratings, faculty evaluations, expert alignment metrics, and standardized patient examinations\u0026mdash;strengthens confidence that the observed effects represent genuine enhancement of underlying reasoning capabilities rather than artifacts of a single assessment method. The higher standardized patient examination scores in the TSM-AL group (80.88\u0026thinsp;\u0026plusmn;\u0026thinsp;3.82 vs 78.71\u0026thinsp;\u0026plusmn;\u0026thinsp;3.90, P\u0026thinsp;=\u0026thinsp;0.025), despite equivalent baseline performance, further support a differential educational impact.\u003c/p\u003e \u003cp\u003eFrom a practical standpoint, these findings offer residency programs a viable pathway for enhancing clinical reasoning instruction within existing resource constraints. The framework leverages LLM capabilities to extend practice opportunities and provide immediate feedback while maintaining faculty oversight through structured supervision protocols\u0026mdash;addressing common challenges in case availability, faculty time, and standardized patient access that the competency-based education literature has identified as barriers to effective clinical reasoning development. Multiple authors have warned that AI-assisted practice gains do not guarantee transfer to independent clinical reasoning ability, with controlled trials typically measuring immediate post-intervention outcomes while long-term independent competency remains uncertain. Fąferek and colleagues specifically identified the lack of in-depth assessment of independent reasoning and the absence of evaluation for transfer to real clinical environments as critical gaps in existing research [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eFrom a practical perspective, these findings suggest that residency programs can leverage structured integration of large language models to enhance clinical reasoning instruction within existing resource constraints. By extending opportunities for case practice and providing timely feedback within defined supervision protocols, the framework helps address common challenges in competency-based training, such as limited case availability, faculty workload pressures, and restricted access to standardized patients. Nonetheless, several limitations warrant consideration. The single-center design within a single surgical specialty limits generalizability, as the reasoning demands and case profiles in orthopedic surgery may differ from those in other disciplines. Outcomes were assessed only at the conclusion of the training period, and the durability of the observed advantages into later stages of residency and independent clinical practice remains unknown. In addition, the rapid evolution of large language model capabilities introduces uncertainty regarding the stability of educational effects across future model architectures and interaction paradigms. Comparative studies examining different model types, prompting strategies, and interaction designs are currently limited and needed to inform best practices [\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e].\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThis study shows that the educational impact of large language models in residency training depends on their integration within structured pedagogical design rather than on their generative capacity alone. When embedded in a deliberate, faculty-supervised clinical reasoning framework, the Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning model supported the internalization of independent and transferable clinical reasoning skills. The persistence of competency gains after withdrawal of model access suggests that strategic use of large language models can foster durable reasoning competence. These findings highlight the potential of intentional AI integration to advance competency-based surgical education while upholding professional judgment and clinical autonomy.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments:\u0026nbsp;\u003c/strong\u003eWe are grateful to the orthopedic surgery residents who participated in this study for their engagement and commitment. We also thank the faculty members and standardized patients at Changzheng Hospital for their contributions to teaching implementation, assessment, and data collection. No external funding was received for this study.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDeclaration of interest statement:\u003c/strong\u003e The authors declare that they have no competing interests. The authors alone are responsible for the content and the writing of this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eNotes on contributors:\u003c/strong\u003e Pengru Wang, Dingyuan Tu, He Li, and Yifei Li contributed equally. Pengru Wang led study design and supervision. Dingyuan Tu and He Li contributed to instructional design and analysis. Bo Li, Yingtian Wang, and Wei Xu provided senior academic oversight.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval and consent to participate:\u0026nbsp;\u003c/strong\u003eThis study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Changzheng Hospital, Navy Military Medical University. Written informed consent was obtained from all participating orthopedic surgery residents, and participation was voluntary with no impact on training evaluation or career progression.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication :\u0026nbsp;\u003c/strong\u003eNot Applicable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u003c/strong\u003e The authors declare that they have no competing interests. The authors alone are responsible for the content and the writing of this article.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003eThis work was supported by the Doctor Assistant Program of the Navy Military Medicine University (SL30) and Education assistant Program (JXPY2023B08).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u0026nbsp;\u003c/strong\u003eThe datasets used and analysed during the current study are available from the corresponding author on reasonable request\u003cbr\u003e\u0026nbsp;\u003cstrong\u003eAuthor Contributions:\u0026nbsp;\u003c/strong\u003ePengru Wang, Dingyuan Tu and He Li contributed equally to this study. Pengru Wang contributed to study conception and design, supervised the educational intervention, and led manuscript drafting. Dingyuan Tu contributed to curriculum design, data analysis, and manuscript preparation. He Li contributed to data acquisition, assessment implementation, and critical manuscript revision. Mengli Chang, Qiying Zhang, and Gan Xu supported participant coordination, data management, and formative assessment implementation. Bo Li, Yingtian Wang, and Wei Xu provided senior academic oversight, contributed to study design refinement, and critically reviewed the manuscript. All authors met the ICMJE criteria for authorship, approved the final version of the manuscript, and agree to be accountable for all aspects of the work.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eShrivastava SR, Bobhate PS, Makade J. Empowering undergraduate medical students with clinical reasoning skills. Journal of Clinical Sciences. 2025;22(3):199-201.\u003c/li\u003e\n\u003cli\u003eCusters E. Training Clinical Reasoning: Historical and Theoretical Background. In: ten Cate O, Custers E, Durning SJ, eds. Principles and Practice of Case-based Clinical Reasoning Education: A Method for Preclinical Students. Cham (CH): Springer Copyright 2018, The Author(s). 2018. p. 21-33.\u003c/li\u003e\n\u003cli\u003eJay R, Davenport C, Patel R. Clinical reasoning-the essentials for teaching medical students, trainees and non-medical healthcare professionals. Br J Hosp Med (Lond). 2024;85(7):1-8.\u003c/li\u003e\n\u003cli\u003eDelavari S, Barzkar F, R MJPR, et al. Teaching and learning clinical reasoning skill in undergraduate medical students: A scoping review. PLoS One. 2024;19(10):e0309606.\u003c/li\u003e\n\u003cli\u003eCahyaningrum YD, Suhoyo Y, Rahayu GR. Facilitating clinical reasoning for medical students in clinical settings: a scoping review. Korean J Med Educ. 2025;37(2):163-86.\u003c/li\u003e\n\u003cli\u003ePlackett RL. Evaluation of an online learning tool to improve medical students\u0026apos; clinical reasoning skills: UCL (University College London); 2019.\u003c/li\u003e\n\u003cli\u003eZhu Y, Zhang J, Fei J, Fang H, Zhang Z. Problem-based learning and case-based learning in clinical practical teaching for gynecology residents: a narrative review. Advances in Medical Education and Practice. 2025:1269-79.\u003c/li\u003e\n\u003cli\u003ePlackett R, Kassianos AP, Mylan S, Kambouri M, Raine R, Sheringham J. The effectiveness of using virtual patient educational tools to improve medical students\u0026rsquo; clinical reasoning skills: a systematic review. BMC medical education. 2022;22(1):365.\u003c/li\u003e\n\u003cli\u003eWang J, Jiang Y, Fu X, et al. Evaluating the impact of interactive video-based case-based learning in clinical medical education: a randomized controlled trial. Frontiers in Medicine. 2025;12:1556018.\u003c/li\u003e\n\u003cli\u003eGasim MS, Ibrahim MH, Abushama WA, Hamed IM, Ali IA. Medical students\u0026rsquo; perceptions towards implementing case-based learning in the clinical teaching and clerkship training. BMC Medical Education. 2024;24(1):200.\u003c/li\u003e\n\u003cli\u003eSingh M, Collins L, Farrington R, et al. From principles to practice: embedding clinical reasoning as a longitudinal curriculum theme in a medical school programme. Diagnosis. 2022;9(2):184-94.\u003c/li\u003e\n\u003cli\u003eK\u0026auml;mmer JE, Hautz WE, Krummrey G, et al. Effects of interacting with a large language model compared with a human coach on the clinical diagnostic process and outcomes among fourth-year medical students: study protocol for a prospective, randomised experiment using patient vignettes. BMJ Open. 2024;14(7):e087469.\u003c/li\u003e\n\u003cli\u003eCahyaningrum YD, Suhoyo Y, Rahayu GR. Facilitating clinical reasoning for medical students in clinical settings: a scoping review. Korean Journal of Medical Education. 2025;37(2):163.\u003c/li\u003e\n\u003cli\u003ePotter L, Jefferies C. Enhancing communication and clinical reasoning in medical education: Building virtual patients with generative AI. Future healthcare journal. 2024;11:100043.\u003c/li\u003e\n\u003cli\u003eZhui L, Yhap N, Liping L, et al. Impact of large language models on medical education and teaching adaptations. JMIR Medical Informatics. 2024;12(1):e55933.\u003c/li\u003e\n\u003cli\u003eBorg A, Jobs B, Huss V, et al. Enhancing clinical reasoning skills for medical students: a qualitative comparison of LLM-powered social robotic versus computer-based virtual patients within rheumatology. Rheumatology International. 2024;44(12):3041-51.\u003c/li\u003e\n\u003cli\u003eHui Z, Zewu Z, Jiao H, Yu C. Application of ChatGPT-assisted problem-based learning teaching method in clinical medical education. BMC Medical Education. 2025;25(1):1-7.\u003c/li\u003e\n\u003cli\u003eBr\u0026uuml;gge E, Ricchizzi S, Arenbeck M, et al. Large language models improve clinical decision making of medical students through patient simulation and structured feedback: a randomized controlled trial. BMC Med Educ. 2024;24(1):1391.\u003c/li\u003e\n\u003cli\u003eWang Z, Fan TT, Li ML, Zhu NJ, Wang XC. Feasibility study of using GPT for history-taking training in medical education: a randomized clinical trial. BMC Med Educ. 2025;25(1):1030.\u003c/li\u003e\n\u003cli\u003eSkryd A, Lawrence K. ChatGPT as a Tool for Medical Education and Clinical Decision-Making on the Wards: Case Study. JMIR Form Res. 2024;8:e51346.\u003c/li\u003e\n\u003cli\u003eFąferek J, Kononowicz AA, Bogutska N, et al. Applying ChatGPT to plan and create a realistic collection of virtual patients for clinical reasoning training. BMC Med Educ. 2025;25(1):1277.\u003c/li\u003e\n\u003cli\u003eEpstein RM. Assessment in medical education. N Engl J Med. 2007;356(4):387-96.\u003c/li\u003e\n\u003cli\u003eFrank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45.\u003c/li\u003e\n\u003cli\u003eGummesson C, Alm S, Cederborg A, et al. Entrustable professional activities (EPAs) for undergraduate medical education - development and exploration of social validity. BMC Med Educ. 2023;23(1):635.\u003c/li\u003e\n\u003cli\u003eHolmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32(8):676-82.\u003c/li\u003e\n\u003cli\u003eGarc\u0026iacute;a-Torres D, Vicente Ripoll MA, Fern\u0026aacute;ndez Peris C, Mira Solves JJ. Enhancing Clinical Reasoning with Virtual Patients: A Hybrid Systematic Review Combining Human Reviewers and ChatGPT. Healthcare (Basel). 2024;12(22).\u003c/li\u003e\n\u003cli\u003eShrivastava S, Bobhate P, Makade J. Empowering undergraduate medical students with clinical reasoning skills. Journal of Clinical Sciences. 2025;22:199-201.\u003c/li\u003e\n\u003cli\u003eOmiye J, Gui H, Rezaei S, Zou J, Daneshjou R. Large language models in medicine: the potentials and pitfalls2023.\u003c/li\u003e\n\u003cli\u003eMontagna M, Chiabrando F, De Lorenzo R, Querini P. Clinical decision support systems during teaching: a hands-on comparison (Preprint). JMIR Medical Education. 2023;11.\u003c/li\u003e\n\u003cli\u003ePlackett R, Kassianos AP, Mylan S, Kambouri M, Raine R, Sheringham J. The effectiveness of using virtual patient educational tools to improve medical students\u0026apos; clinical reasoning skills: a systematic review. BMC Med Educ. 2022;22(1):365.\u003c/li\u003e\n\u003cli\u003eHudon A, Phan V, Charlin B, Wittmer R. Teaching Clinical Reasoning in Health Care Professions Learners Using AI-Generated Script Concordance Tests: Mixed Methods Formative Evaluation. JMIR Form Res. 2025;9:e76618.\u003c/li\u003e\n\u003cli\u003eCusters E. Training Clinical Reasoning: Historical and Theoretical Background. 2018. p. 21-33.\u003c/li\u003e\n\u003cli\u003eHui Z, Zewu Z, Jiao H, Yu C. Application of ChatGPT-assisted problem-based learning teaching method in clinical medical education. BMC Medical Education. 2025;25. \u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTable 1. Baseline Characteristics and Pre-training Assessment Scores\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"595\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eVariable\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;TSM-AL\u0026nbsp;\u003cbr\u003e\u0026nbsp;Group (n= 34)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTraditional Group (n= 34)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eP value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eAge (years)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003e28.18 \u0026plusmn; 3.41\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003e27.99 \u0026plusmn; 3.78\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.667\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eGender (Male/Female)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003e22 (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003e18 (%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eTraining year\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003ePGY-1 11 (32.35)\u0026nbsp;\u003cbr\u003e\u0026nbsp; PGY-2 12 (35.29)\u003cbr\u003e\u0026nbsp; PGY-3 11 (32.35)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003ePGY-1 10 (29.41)\u0026nbsp;\u003cbr\u003e\u0026nbsp; PGY-2 15 (44.12)\u003c/p\u003e\n \u003cp\u003ePGY-3 9 (26.47）\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.748\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eDegree\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003eBachelor 14 (41.18)\u003cbr\u003e\u0026nbsp;Master 7 (20.59)\u003c/p\u003e\n \u003cp\u003eDoctor\u0026nbsp;13 (38.24)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003eBachelor 10 (29.41)\u003cbr\u003e\u0026nbsp;Master 12 (35.29)\u003c/p\u003e\n \u003cp\u003eDoctor 12 (35.29)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.364\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eStandardized patient score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003e71.56 \u0026plusmn; 4.23\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003e71.15 \u0026plusmn; 2.69\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.349\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eClinical reasoning score (0\u0026ndash;10)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003e3.47 \u0026plusmn; 1.61\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003e4.35 \u0026plusmn; 2.14\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.095\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eEPA level (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003eLevel 1 4(11.76)\u003cbr\u003e\u0026nbsp;Level 2 12(35.29)\u003cbr\u003e\u0026nbsp;Level 3 9(26.47)\u003cbr\u003e\u0026nbsp;Level 4 8(23.53)\u003cbr\u003e\u0026nbsp;Level 5 1(2.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003eLevel 1 11(32.35)\u003cbr\u003e\u0026nbsp;Level 2 8(23.53)\u003cbr\u003e\u0026nbsp;Level 3 3(8.82)\u003cbr\u003e\u0026nbsp;Level 4 11(32.35)\u003cbr\u003e\u0026nbsp;Level 5 1(2.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.110\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 138px;\"\u003e\n \u003cp\u003eGlobal faculty rating (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 183px;\"\u003e\n \u003cp\u003eLevel 1 7(20.59)\u003cbr\u003e\u0026nbsp;Level 2 9(26.47)\u0026nbsp;\u003cbr\u003e\u0026nbsp;Level 3 8(23.53)\u003cbr\u003e\u0026nbsp;Level 4 9(26.47)\u003cbr\u003e\u0026nbsp;Level 5 1(2.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 161px;\"\u003e\n \u003cp\u003eLevel 1 5(14.71)\u003cbr\u003e\u0026nbsp;Level 2 10(29.41)\u003cbr\u003e\u0026nbsp;Level 3 7(20.59)\u003cbr\u003e\u0026nbsp;Level 4 12(35.29)\u003cbr\u003e\u0026nbsp;Level 5 0(0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 113px;\"\u003e\n \u003cp\u003e0.758\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 2. Comparison of Formative Assessment Scores During Training\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"595\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAssessment Domain\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTSM-AL Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTraditional Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eP value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u0026nbsp;\u003cbr\u003e\u0026nbsp;assessment-1(2w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e2.00 \u0026plusmn; 1.35\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e2.03 \u0026plusmn; 1.40\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.931\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e0.94 \u0026plusmn; 0.87\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u0026nbsp;\u003cbr\u003e\u0026nbsp;assessment-2(4w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e2.18 \u0026plusmn; 1.32\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e2.09 \u0026plusmn; 1.50\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.800\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1.12 \u0026plusmn; 0.83\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eassessment-3(6w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e2.29 \u0026plusmn; 1.56\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e1.94 \u0026plusmn; 1.28\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.320\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1.15 \u0026plusmn; 0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eassessment-4(12w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e3.26 \u0026plusmn; 1.54\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e2.68 \u0026plusmn; 1.28\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.096\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e1.41 \u0026plusmn; 0.91\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003eassessment-5(14w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e3.53\u0026plusmn; 1.36\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e2.68 \u0026plusmn; 1.49\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.018\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e2.26 \u0026plusmn; 0.92\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eFormative\u003c/strong\u003e\u003c/p\u003e\n \u003cp\u003e\u003cstrong\u003e\u0026nbsp;assessment-6(16w)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eDiagnostic pathway completeness (0\u0026ndash;6)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e3.35 \u0026plusmn; 1.43\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e2.37 \u0026plusmn; 1.51\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e0.009\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 170px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;4)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 142px;\"\u003e\n \u003cp\u003e2.06 \u0026plusmn; 0.97\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 151px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 132px;\"\u003e\n \u003cp\u003e\u0026mdash;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3. Final Assessment Outcomes: LLM-assisted Examination\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"3\" cellpadding=\"0\" width=\"444\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eOutcome\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTSM-AL Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eClinical reasoning score (0\u0026ndash;10)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e5.85 \u0026plusmn; 1.99\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eEPA level (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003eLevel 1 6(17.65)\u003cbr\u003e\u0026nbsp;Level 2 6(17.65)\u003cbr\u003e\u0026nbsp;Level 3 7(20.59)\u003cbr\u003e\u0026nbsp;Level 4 12(35.29)\u003cbr\u003e\u0026nbsp;Level 5 3(8.82)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eGlobal faculty rating (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003eLevel 1 4(11.77)\u003cbr\u003e\u0026nbsp;Level 2 6(17.65)\u0026nbsp;\u003cbr\u003e\u0026nbsp;Level 3 10(29.41)\u003cbr\u003e\u0026nbsp;Level 4 8(23.53)\u003cbr\u003e\u0026nbsp;Level 5 6(17.65)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eAppropriate use of LLM (0\u0026ndash;8)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e4.82 \u0026plusmn; 1.60\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eCore problem aligned with expert\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e21(61.77%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003ePrimary diagnosis aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e22(64.71%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003e\u0026ge;2 of top 3 diagnoses aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e20(58.82%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 219px;\"\u003e\n \u003cp\u003eInitial investigation aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 220px;\"\u003e\n \u003cp\u003e21(61.77%)\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 4. Final Assessment Outcomes:\u0026nbsp;\u003cbr\u003e\u0026nbsp;Non-LLM (Independent Performance) Examination\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"3\" cellpadding=\"0\" width=\"605\"\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eOutcome\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eTSM-AL Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eTraditional Group\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e\u003cstrong\u003eP value\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eClinical reasoning score (0\u0026ndash;10)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e7.32 \u0026plusmn; 2.30\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e5.12 \u0026plusmn; 1.66\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.000\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eEPA level (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLevel 1 3(11.76)\u003cbr\u003e\u0026nbsp;Level 2 7(35.29)\u003cbr\u003e\u0026nbsp;Level 3 (26.47)\u003cbr\u003e\u0026nbsp;Level 4 8(23.53)\u003cbr\u003e\u0026nbsp;Level 5 1(2.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLevel 1 10(29.41)\u003cbr\u003e\u0026nbsp;Level 2 8(23.53)\u003cbr\u003e\u0026nbsp;Level 3 9(8.82)\u003cbr\u003e\u0026nbsp;Level 4 4(32.35)\u003cbr\u003e\u0026nbsp;Level 5 2(2.94)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.034\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eGlobal faculty rating (1\u0026ndash;5)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLevel 1 3(8.82)\u003cbr\u003e\u0026nbsp;Level 2 6(17.65)\u0026nbsp;\u003cbr\u003e\u0026nbsp;Level 3 8(23.53)\u003cbr\u003e\u0026nbsp;Level 4 9(26.47)\u003cbr\u003e\u0026nbsp;Level 5 8(23.53)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003eLevel 1 9(26.47)\u003cbr\u003e\u0026nbsp;Level 2 8(23.53)\u003cbr\u003e\u0026nbsp;Level 3 10(29.41)\u003cbr\u003e\u0026nbsp;Level 4 7(20.59)\u003cbr\u003e\u0026nbsp;Level 5 0(0)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.019\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eCore problem aligned with expert\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e23(67.65%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e15(44.12%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.042\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003ePrimary diagnosis aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e18(53.94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e16(44.12%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.804\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003e\u0026ge;2 of top 3 diagnoses aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e25(73.53%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e17(50.00%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.046\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eInitial investigation aligned\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e16(47.06%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e18(52.94%)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.808\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cp\u003eStandardized patient score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e80.88 \u0026plusmn; 3.82\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e78.71 \u0026plusmn; 3.90\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cp\u003e0.025\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Clinical reasoning, Competency-based medical education, Residency training, Large language models, Artificial intelligence in medical education, Entrustable professional activities, Self-directed learning","lastPublishedDoi":"10.21203/rs.3.rs-8818923/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8818923/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eThis study evaluated whether a structured pedagogical framework integrating large language models (LLMs) into residency training could develop clinical reasoning competencies that transfer to independent performance.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e \u003cp\u003eIn this prospective randomized controlled trial, residents were assigned to Teacher\u0026ndash;Student\u0026ndash;Machine Interaction Autonomous Learning (TSM-AL) group or traditional teaching group. The TSM-AL framework positioned LLMs as supervised cognitive auxiliaries within a five-step case analysis process featuring controlled information release, predefined reasoning tasks, and structured utilization guidelines. Six formative assessments were conducted over 16 weeks, followed by summative examinations under both LLM-assisted and unassisted conditions. Outcomes included clinical reasoning scores, entrustable professional activity (EPA) levels, global faculty ratings, expert-aligned diagnostic accuracy, and standardized patient examination performance.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eSignificant between-group differences in diagnostic pathway completeness emerged at week 14 (3.53\u0026thinsp;\u0026plusmn;\u0026thinsp;1.36 vs 2.68\u0026thinsp;\u0026plusmn;\u0026thinsp;1.49, P\u0026thinsp;=\u0026thinsp;0.018) and week 16 (3.35\u0026thinsp;\u0026plusmn;\u0026thinsp;1.43 vs 2.37\u0026thinsp;\u0026plusmn;\u0026thinsp;1.51, P\u0026thinsp;=\u0026thinsp;0.009). In the unassisted final examination, the TSM-AL group demonstrated significantly higher clinical reasoning scores (7.32\u0026thinsp;\u0026plusmn;\u0026thinsp;2.30 vs 5.12\u0026thinsp;\u0026plusmn;\u0026thinsp;1.66, P\u0026thinsp;\u0026lt;\u0026thinsp;0.001). EPA distributions differed significantly (P\u0026thinsp;=\u0026thinsp;0.034), with fewer TSM-AL residents requiring complete supervision (11.76% vs 29.41%). Global faculty ratings also differed significantly (P\u0026thinsp;=\u0026thinsp;0.019), with 23.53% of TSM-AL residents achieving Level 5 compared to none in the traditional group. Expert alignment for core problem identification (67.65% vs 44.12%, P\u0026thinsp;=\u0026thinsp;0.042) and differential diagnosis matching (73.53% vs 50.00%, P\u0026thinsp;=\u0026thinsp;0.046) favored the TSM-AL group. Standardized patient examination scores were significantly higher in the TSM-AL group (80.88\u0026thinsp;\u0026plusmn;\u0026thinsp;3.82 vs 78.71\u0026thinsp;\u0026plusmn;\u0026thinsp;3.90, P\u0026thinsp;=\u0026thinsp;0.025). Notably, TSM-AL residents performed better without LLM assistance than with assistance.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e \u003cp\u003eThe TSM-AL framework significantly enhanced clinical reasoning competencies that transferred to independent performance, demonstrating that structured LLM integration develops autonomous reasoning rather than fostering technological dependency.\u003c/p\u003e","manuscriptTitle":"Teacher–Student–Machine Interaction Autonomous Learning: A Structured LLM-Integrated Framework for Developing Independent Clinical Reasoning in Residency Training","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2026-04-26 15:37:16","doi":"10.21203/rs.3.rs-8818923/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-05-04T16:24:16+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-30T18:38:00+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-22T15:33:38+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"99530526461434160261274900126573735309","date":"2026-04-22T15:17:20+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"150399590557023404962822330837509707279","date":"2026-04-18T18:30:19+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2026-04-16T11:47:01+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2026-04-08T11:01:22+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2026-02-27T16:32:41+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2026-02-25T09:43:11+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2026-02-25T09:35:05+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"27275a3a-1fe8-4c5a-ad48-0b95510f5b2f","owner":[],"postedDate":"April 26th, 2026","published":true,"recentEditorialEvents":[{"type":"decision","content":"Revision requested","date":"2026-05-04T16:24:16+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-04-30T18:38:00+00:00","index":52,"fulltext":""}],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[],"tags":[],"updatedAt":"2026-05-04T16:38:56+00:00","versionOfRecord":[],"versionCreatedAt":"2026-04-26 15:37:16","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8818923","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8818923","identity":"rs-8818923","version":["v1"]},"buildId":"XKTyCvWXoU3ODBz1xrDgd","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}