Bridging the Gap Between Observation and Rating in the Mini-CEX: Exploring a Task- and-Process Approach

Bridging the Gap Between Observation and Rating in the Mini-CEX: Exploring a Task- and-Process Approach | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Bridging the Gap Between Observation and Rating in the Mini-CEX: Exploring a Task- and-Process Approach Qiankuan Wen, Jun Wang, Xin Wang, Na Li, Hui Li, Kang Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7412906/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 5 You are reading this latest preprint version Abstract Purpose At present, rater idiosyncrasy in the Mini-CEX remains an issue that needs to be addressed. This study proposes a "task and process" method to reduce inter-rater variation and simplify the rating process. Methods The "task and process" method consists of two steps. While conducting a Mini-CEX, the rater first determines the task associated with each item and whether the resident completes the task. The task is then broken down into major and minor components. Based on how the task is completed, a task score is assigned. Second, the rater observes whether the resident's behavior aligns with the ABIM's Mini-CEX guidelines and assigns a process score. The task score and process score are combined to generate a final score for each item. The inter-rater intraclass correlation coefficient (ICC) was calculated for each item. Results In 2024, six physicians rated seven residents across seven Mini-CEX encounters. ICCs were calculated for each item, ranging from 0.700 to 0.904, indicating moderate to excellent reliability. Conclusions The "task and process" method offers a structured framework for raters in the Mini-CEX, helping bridge the gap between observation and evaluation. Further studies are needed to refine and validate this approach. Mini-CEX Rater idiosyncrasy Inter-rater variation Task and process method Intraclass correlation coefficient Introduction The Mini-Clinical Evaluation Exercise (Mini-CEX) is an assessment tool developed by the American Board of Internal Medicine (ABIM) to evaluate the clinical competencies of medical students and residents. It involves observing a routine clinical encounter between a resident and a patient in a real-world setting. After the observation, the evaluator asks the resident diagnostic or therapeutic questions related to the case and subsequently assesses their clinical competence across seven domains: medical interviewing, physical examination, humanistic qualities/professionalism, clinical judgment, counseling skills, organization/efficiency, and overall clinical competence. The evaluator then provides feedback to the resident. Typically, the observation takes about 15–20 minutes, followed by 5–10 minutes of feedback (Norcini et al., 2003 ). The Mini-CEX has been widely implemented in various clinical settings, including inpatient, outpatient, and emergency departments, serving both formative and summative assessment purposes. Its feasibility, validity, and reliability have been demonstrated in numerous studies and are generally accepted by both faculty and trainees (Sharma et al., 2022 ; Zaki et al., 2023 ; Berendonk et al., 2018 ). However, there is no universally accepted "gold standard" for judging performance in the Mini-CEX (Norcini et al., 2003 ). After observing a resident-patient encounter, the examiner must translate their observations into ratings and feedback. The ABIM provides implementation guidelines for the Mini-CEX ( https://www.abim.org/program-directors-administrators/assessment-toolsmini-cex ), recommending a two-step approach for using the nine-point scale: first, determine whether the performance is satisfactory, unsatisfactory, or superior; second, select the rating within that category that best reflects the observed performance. However, these guidelines describe idealized behaviors, such as "facilitates accurate collection of a patient’s history" or "effectively uses questions/directions to obtain accurate information." Interpreting and applying these abstract criteria pose significant challenges for examiners, particularly when deciding between levels of performance. As a result, substantial inter-rater variability can occur. Although several studies have addressed this issue (Rogausch et al., 2015 ; Gittinger et al., 2022 ), the problem remains unresolved. Specifically, how should an evaluator judge a resident’s performance based on their perception of these guidelines? A single resident-patient encounter evaluated by different examiners may yield significantly different scores due to individual interpretations of the standards. Currently, there is no standardized rating criterion. Previous research has shown that evaluators engage in complex cognitive processes during the Mini-CEX, involving observation, judgment, and rating. These assessments are influenced by various factors, including the clinical environment, the resident’s behavior, patient characteristics, and the evaluator’s own clinical and teaching experience (Kogan et al., 2011 ). Consequently, bias is inevitable, and controlling this bias is a pressing concern (Lee et al., 2017 ). In this study, we propose a “task and process” method for evaluators to use when conducting the Mini-CEX. We aim to improve the feasibility and reliability of the Mini-CEX by providing a more structured and transparent approach to assessment. Methods Design In a clinical encounter, a resident’s task is to collect data for the medical record, then formulate a diagnosis and treatment plan based on that information. During a Mini-CEX, the evaluator may imagine how they would complete the clinical record based on the same encounter—asking: Is the information gathered sufficient for clinical decision-making? Thus, the concept of "task" becomes the foundation for assessment. The “task and process” method is derived from this principle. For the Mini-CEX’s seven domains, specific tasks were identified. First, each domain was assigned a primary task. Then, each task was divided into major and minor components ( see Table 1 ). The scoring rubric was as follows: Major task completed: Score of 5 (satisfactory) Minor task completed: Score of 4 (marginal) Both major and minor tasks completed: Score of 6 (high satisfactory) Neither completed: Score of 3 (unsatisfactory) Next, the evaluator assessed whether the process matched the ABIM guidelines. Process scores were assigned based on the degree of alignment: > 2/3 of guideline behaviors matched: Score of 3 1/3–2/3 matched: Score of 2 < 1/3 matched: Score of 1 The final score for each item was the sum of the task score and the process score. For the item "Overall Clinical Competence", the score was calculated differently. Since the process components are already assessed in the other six items, only the task scores were used here. The calculation was based on three core domains—Medical Interviewing, Physical Examination, and Clinical Judgment: If all three scores > 5 → Overall competence score = 9 If two scores > 5 → Score = 8 If one score > 5 → Score = 7 If all three scores ≥ 4 → Score = 6 If any of the three scores < 4 → Use the highest score among the six domains as the overall competence score. Ethical Considerations The study protocol was approved by the Review Board of China-Japan Friendship Hospital. Oral informed consent was obtained from all participants. Participants Evaluators Six evaluators (raters) were faculty members from China-Japan Friendship Hospital. All were experienced in conducting and evaluating Mini-CEX and had extensive backgrounds in clinical teaching. The “task and process” method was co-developed by this group. Residents Seven residents participated while on clinical rotation. They were informed in advance that five additional evaluators would observe their clinical encounter but would not provide feedback. Their scores would not be recorded in the residents’ official evaluations. Oral consent was obtained before participation. Patients Seven new patients participated in the Mini-CEX encounters. Informed consent was obtained from each patient before the evaluation. Statistics To assess inter-rater reliability, intraclass correlation coefficients (ICC) and their 95% confidence intervals were calculated using SPSS version 23 (SPSS Inc., Chicago). A two-way random-effects model with consistency type was used, based on single measurements (k = 6 raters). The ICC values were interpreted as follows: 0.90: Excellent reliability. Results Participants Table 3 presents information about the study participants. Six evaluators, each from a different department, participated in the study. Between March 2024 and December 2024, seven Mini-CEX encounters were conducted: 2 in the Department of Emergency Medicine 2 in the Endocrinology Department 1 in the Second Department of Geriatrics 1 in the Department of Acupuncture and Moxibustion 1 in the Department of Neurology Seven patients (4 males and 3 females), with an average age of 54 ± 8.0 years, were recruited. Their diagnoses included: upper gastrointestinal hemorrhage, severe pneumonia, diabetes mellitus, hypothyroidism, chronic heart failure, neck-angle pain, and Guillain-Barré Syndrome ( see Table 2 ). Ratings Across all evaluations, the minimum score was 3, assigned by five evaluators across six different items (excluding overall clinical competence). The maximum score was 9, awarded by Evaluator 5 to Resident 4 in the domains of clinical judgment and overall clinical competence. Intraclass correlation coefficients (ICCs) were calculated separately for each of the seven Mini-CEX items ( see Table 3 ): The lowest ICC was 0.700 for the Physical Examination item, followed by Counseling Skills with an ICC of 0.746, both indicating moderate reliability. The highest ICC was for Overall Clinical Competence , with a value of 0.904, indicating excellent reliability. The remaining four items had ICCs between 0.75 and 0.90, reflecting good reliability. These results suggest that the "task and process" method leads to consistent scoring across raters, particularly for composite or integrative items like overall clinical competence. Table 1 Information of the tasks and their breakdowns Major task Minor task Medical interviewing Chief complaint and history of present illness Past medical history, family history, personal and social history Physical examination Chief complaint correlated focused physical examination, find positive signs Differential negative signs Humanistic qualities/professionalism Doctor’s perspective: doesn’t violate medical practice Patient’s perspective: not awkward, Clinical judgment Diagnosis, reasonable differential diagnosis Further test and treatment plan Counselling skills Explains rationale for test/treatment Obtains a patient’s consent Organization/efficiency Complete medical interviewing, physical examination and clinical judgement Less than 30 minutes Overall clinical competency Medical interviewing, physical examination, clinical judgement Humanistic qualities/professionalism, counselling skills, organization/efficiency Table 2 Information of the participants in this study Evaluator Residents Patients Gender Department Gender PGY gender Year old Department Diagnosis F Pediatric M 2 F 64 Endocrine Diabetes mellitus F The 2nd Geriatric F 3 M 54 Emergency Upper gastrointestinal hemorrhage M Emergency M 2 M 46 Endocrine Hypothyroidism F Endocrine F 2 M 62 The 2nd Geriatric Chronic heart failure F Acupuncture and Moxibustion F 3 F 42 Neurology Guillain-barre syndrome M Neurology F 3 F 57 Emergency Severe pneumonia M 3 M 56 CTM* Neck-angle pain • CTM: the Department of Chinese Traditional Medicine Table 3 Intraclass correlation coefficients Intraclass correlation 95% confidence interval Lower bound Upper bound Interviewing 0.866 0.682 0.70 Physical examination 0.700 0.42 0.926 Humanistic qualities/Professionalism 0.876 0.707 0.974 Clinical judgement 0.86 0.670 0.970 Counseling 0.746 0.478 0.940 Organization and efficiency 0.856 0.664 0.970 Overall clinical competence 0.904 0.760 0.98 Discussion In medical education, producing a clinical record is a vital component of developing a resident’s clinical competence. A clear, concise, and comprehensive record serves as the foundation for patient assessment and treatment planning. After a patient encounter, residents are expected to document their findings and generate a clinical record. In this study, we incorporated the concept of the clinical record into the Mini-CEX, viewing it as a critical task in assessing resident performance. We introduced the "task and process" method as a dual-step approach to rating: Task completion is used as the first indicator of performance—determining whether the resident completed the expected major or minor clinical task. This step corresponds to the ABIM's first recommended step of determining whether performance is satisfactory, unsatisfactory, or superior. Process evaluation assesses how closely the resident’s behavior aligns with ABIM’s guidelines, contributing a process score. This aligns with ABIM’s second step of refining the rating within the selected performance category. By combining task and process scores, a valid and structured final score is achieved. The “task and process” method acts as a bridge between observation and evaluation, while also linking training to demonstrated competency. Additionally, breaking tasks down into major and minor components helps evaluators apply the Mini-CEX more accurately and systematically. Our results show that the "task and process" method demonstrates moderate to excellent inter-rater reliability. Among all items, Physical Examination showed the lowest reliability (ICC = 0.700, 95% CI: 0.42–0.926), followed by Counseling Skills (ICC = 0.746, 95% CI: 0.478–0.940), both indicating moderate reliability. This may be due to the vague or subjective nature of the task and process definitions for these domains. For example, the physical examination task requires three subjective judgments: identifying the chief complaint, distinguishing major and minor tasks, and comparing observed behavior with guidelines. This complexity introduces variability in scoring. In contrast, the Overall Clinical Competence item showed the highest reliability (ICC = 0.904, 95% CI: 0.76–0.98), likely due to the simplicity of its rating rules. The remaining items showed good reliability, with ICCs ranging from 0.856 to 0.876 (95% CI: 0.664–0.974). These results suggest that the "task and process" method simplifies the rating process while maintaining consistency and clarity for evaluators. Limitations This study has several limitations. First , the “task and process” method is a novel concept introduced by a small group of experienced clinicians. While they are highly familiar with the Mini-CEX, the method may have flaws and requires further refinement. It has only been tested in a single institution, and its reliability and generalizability need to be validated in broader settings. Second , the method may restrict evaluator flexibility during both rating and feedback, which could limit nuanced assessments based on individual encounters. Third , for the domain of Humanistic Qualities/Professionalism , the minor task involves assessing whether the resident considers the patient’s perspective. This poses a challenge, as evaluators are typically trained to view encounters from a physician’s standpoint. Shifting to a patient’s perspective during rating may be difficult and could impact reliability. These limitations should be addressed in future studies to further improve the method. Conclusion The “task and process” method offers a simplified and structured approach for evaluators conducting the Mini-CEX. It provides a clear framework for assessing resident-patient encounters and helps bridge the gap between observation and rating. The method demonstrated moderate to excellent inter-rater reliability, suggesting its potential value in clinical training. However, further research is needed to optimize and validate the approach across different settings and specialties. Declarations Acknowledgments : The authors express their sincere gratitude to the patients who participated in this study. Author contributions: QW, JW, XW, NL, KW performed data collection and analysis. All authors contributed to the study’s conceptualization, result interpretation, and manuscript draft. Funding : none. Availability of data and materials: The dataset supporting the conclusions of this article is included within the article, and is available from the corresponding author on reasonable request. Ethical approval and consent to participate : All clinical procedures and study protocols were approved by the Ethics Committee of the China-Japan Friendship Hospital. Written informed consent was obtained from all patients prior to the start of the study. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests. Clinical trial number: not applicable References Berendonk C, Rogausch A, Gemperli A, Himmel W. (2018) Variability and dimensionality of students' and supervisors' mini-CEX scores in undergraduate medical clerkships - a multilevel factor analysis. BMC Med Educ. 2018 May 8;18(1):100. doi: 10.1186/s12909-018-1207-1. PMID: 29739387; PMCID: PMC5941409. Gingerich A, van der Vleuten CP, Eva KW, Regehr G. (2014) More consensus than idiosyncrasy: Categorizing social judgments to examine variability in Mini-CEX ratings. Acad Med. 2014 Nov;89(11):1510-9. doi: 10.1097/ACM.0000000000000486. PMID: 25250753. Gittinger FP, Lemos M, Neumann JL, Förster J, Dohmen D, Berke B, Olmeo A, Lucas G, Jonas SM. (2022) Interrater reliability in the assessment of physiotherapy students. BMC Med Educ. 2022 Mar 16;22(1):186. doi: 10.1186/s12909-022-03231-y. PMID: 35296313; PMCID: PMC8928589. Kogan JR, Conforti L, Bernabeo E, Iobst W, Holmboe E. (2011) Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ. 2011 Oct;45(10):1048-60. doi: 10.1111/j.1365-2923.2011.04025.x. PMID: 21916943. Lee V, Brain K, Martin J. (2017) Factors Influencing Mini-CEX Rater Judgments and Their Practical Implications: A Systematic Literature Review. Acad Med. 2017 Jun;92(6):880-887. doi: 10.1097/ACM.0000000000001537. PMID: 28030422. Norcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003 Mar 18;138(6):476-81. doi: 10.7326/0003-4819-138-6-200303180-00012. PMID: 12639081. Rogausch A, Beyeler C, Montagne S, Jucker-Kupper P, Berendonk C, Huwendiek S, Gemperli A, Himmel W. (2015) The influence of students' prior clinical skills and context characteristics on mini-CEX scores in clerkships--a multilevel analysis. BMC Med Educ. 2015 Nov 25;15:208. doi: 10.1186/s12909-015-0490-3. PMID: 26608836; PMCID: PMC4658793. Sharma R, Gupta T, Haidery TH, Sinha S, Kumar A. (2022) Current Trends in Mini-Clinical Evaluation Exercise in Medical Education: A Bibliometric Analysis. Cureus. 2022 Dec 30;14(12):e33121. doi: 10.7759/cureus.33121. PMID: 36721606; PMCID: PMC9884390. Zaki HA, Yigit Y, Shaban E, Shaban AE, Elmoheen A, Bashir K, Basharat K, Ali M, Hamdi Alkahlout B. (2023) The Utility of the Mini-Clinical Evaluation Exercise (Mini-CEX) in the Emergency Department: A Systematic Review and Meta-Analysis Evaluating the Readability, Feasibility, and Acceptability of Mini-CEX Utilization. Cureus. 2023 Aug 31;15(8):e44443. doi: 10.7759/cureus.44443. PMID: 37791168; PMCID: PMC10544045. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviewers invited by journal 25 Sep, 2025 Editor invited by journal 03 Sep, 2025 Editor assigned by journal 01 Sep, 2025 Submission checks completed at journal 01 Sep, 2025 First submitted to journal 19 Aug, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7412906","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":524894779,"identity":"c0e56e0f-97ef-4ed2-88cf-436201cbbfdf","order_by":0,"name":"Qiankuan Wen","email":"","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":false,"prefix":"","firstName":"Qiankuan","middleName":"","lastName":"Wen","suffix":""},{"id":524894782,"identity":"58852e2d-a45f-450b-b8a6-487dd877c597","order_by":1,"name":"Jun Wang","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA7klEQVRIiWNgGAWjYLACCTDJA8QVEnL8JGo5Y2Es2UC8XUAtjG0ViRsIaTE4fvbwC8u2O/bmM3KPSfPOk2DcwMD88NENfFrO5KVZSLY9Y5a5kZcmzbtNgtmcgc3YOAePFrMDOWYGkm2H2SQkcsxAWtgsG3jYpPFqOf8GrIUHomWOBI/BAUJabuQYPwBqkYBoaZCQIKjF/sYbMwaJc4cNJHjeGFvOOSZhINlMwC+S/TnGnyXKDttLsOcY3nhTU1ffz9788DE+LUDAJg2OSoEEFilQbDIw41cOVvLxA4jiP8D88Qdh1aNgFIyCUTACAQB680YJXY8LOQAAAABJRU5ErkJggg==","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":true,"prefix":"","firstName":"Jun","middleName":"","lastName":"Wang","suffix":""},{"id":524894783,"identity":"f6e38a3a-921c-44b7-90b2-450e46af30c0","order_by":2,"name":"Xin Wang","email":"","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":false,"prefix":"","firstName":"Xin","middleName":"","lastName":"Wang","suffix":""},{"id":524894787,"identity":"e6836898-ba61-421e-8e79-9a395d730f73","order_by":3,"name":"Na Li","email":"","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":false,"prefix":"","firstName":"Na","middleName":"","lastName":"Li","suffix":""},{"id":524894789,"identity":"fae3d9f4-8e4c-4e22-b31d-dbfac1fd473b","order_by":4,"name":"Hui Li","email":"","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":false,"prefix":"","firstName":"Hui","middleName":"","lastName":"Li","suffix":""},{"id":524894790,"identity":"0c8a5007-e9b9-481a-bb67-3a5c920b3851","order_by":5,"name":"Kang Wang","email":"","orcid":"","institution":"China-Japan Friendship Hospital","correspondingAuthor":false,"prefix":"","firstName":"Kang","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2025-08-20 03:23:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-7412906/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7412906/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":92970379,"identity":"80233b57-b4d9-4db1-bfab-d683ee4f4d33","added_by":"auto","created_at":"2025-10-07 16:39:38","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":39520,"visible":true,"origin":"","legend":"","description":"","filename":"submission.docx","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/d8c931e20b73a58ca89a0d9f.docx"},{"id":92970416,"identity":"32661338-8bc8-4170-956a-a2df333afbe8","added_by":"auto","created_at":"2025-10-07 16:39:45","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":7055,"visible":true,"origin":"","legend":"","description":"","filename":"82b953378e1646b6b2875bd72b166a0d.json","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/5bfad10ed7ef8f8ad734da28.json"},{"id":92970412,"identity":"bb5968fc-8827-4fc8-9700-16f8cec7d520","added_by":"auto","created_at":"2025-10-07 16:39:42","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":55708,"visible":true,"origin":"","legend":"","description":"","filename":"82b953378e1646b6b2875bd72b166a0d1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/ac472e302ce437c555d4ccb7.xml"},{"id":92970432,"identity":"8a3b9086-ea4c-4aee-ac67-a0556f36db97","added_by":"auto","created_at":"2025-10-07 16:39:47","extension":"xml","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":52284,"visible":true,"origin":"","legend":"","description":"","filename":"82b953378e1646b6b2875bd72b166a0d1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/048ec33f93e4a440e4d4bdfd.xml"},{"id":92970410,"identity":"9e2c15a9-e71a-469a-9cd5-b88f641a2366","added_by":"auto","created_at":"2025-10-07 16:39:41","extension":"html","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":62299,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/7c127e387a52d9ce2c8ec0b5.html"},{"id":92970586,"identity":"e0e03448-5497-4863-87b9-a4664b8ccd4d","added_by":"auto","created_at":"2025-10-07 16:40:09","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":571924,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7412906/v1/bb3d802c-1c53-4879-af62-fcd63e2dd80e.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Bridging the Gap Between Observation and Rating in the Mini-CEX: Exploring a Task- and-Process Approach","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe Mini-Clinical Evaluation Exercise (Mini-CEX) is an assessment tool developed by the American Board of Internal Medicine (ABIM) to evaluate the clinical competencies of medical students and residents. It involves observing a routine clinical encounter between a resident and a patient in a real-world setting. After the observation, the evaluator asks the resident diagnostic or therapeutic questions related to the case and subsequently assesses their clinical competence across seven domains: medical interviewing, physical examination, humanistic qualities/professionalism, clinical judgment, counseling skills, organization/efficiency, and overall clinical competence. The evaluator then provides feedback to the resident. Typically, the observation takes about 15\u0026ndash;20 minutes, followed by 5\u0026ndash;10 minutes of feedback (Norcini et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2003\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eThe Mini-CEX has been widely implemented in various clinical settings, including inpatient, outpatient, and emergency departments, serving both formative and summative assessment purposes. Its feasibility, validity, and reliability have been demonstrated in numerous studies and are generally accepted by both faculty and trainees (Sharma et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Zaki et al., \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2023\u003c/span\u003e; Berendonk et al., \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e2018\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eHowever, there is no universally accepted \"gold standard\" for judging performance in the Mini-CEX (Norcini et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2003\u003c/span\u003e). After observing a resident-patient encounter, the examiner must translate their observations into ratings and feedback. The ABIM provides implementation guidelines for the Mini-CEX (\u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.abim.org/program-directors-administrators/assessment-toolsmini-cex\u003c/span\u003e\u003cspan address=\"https://www.abim.org/program-directors-administrators/assessment-toolsmini-cex\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e), recommending a two-step approach for using the nine-point scale: first, determine whether the performance is satisfactory, unsatisfactory, or superior; second, select the rating within that category that best reflects the observed performance. However, these guidelines describe idealized behaviors, such as \"facilitates accurate collection of a patient\u0026rsquo;s history\" or \"effectively uses questions/directions to obtain accurate information.\" Interpreting and applying these abstract criteria pose significant challenges for examiners, particularly when deciding between levels of performance. As a result, substantial inter-rater variability can occur.\u003c/p\u003e\u003cp\u003eAlthough several studies have addressed this issue (Rogausch et al., \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2015\u003c/span\u003e; Gittinger et al., \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), the problem remains unresolved. Specifically, how should an evaluator judge a resident\u0026rsquo;s performance based on their perception of these guidelines? A single resident-patient encounter evaluated by different examiners may yield significantly different scores due to individual interpretations of the standards. Currently, there is no standardized rating criterion.\u003c/p\u003e\u003cp\u003ePrevious research has shown that evaluators engage in complex cognitive processes during the Mini-CEX, involving observation, judgment, and rating. These assessments are influenced by various factors, including the clinical environment, the resident\u0026rsquo;s behavior, patient characteristics, and the evaluator\u0026rsquo;s own clinical and teaching experience (Kogan et al., \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2011\u003c/span\u003e). Consequently, bias is inevitable, and controlling this bias is a pressing concern (Lee et al., \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2017\u003c/span\u003e).\u003c/p\u003e\u003cp\u003eIn this study, we propose a \u0026ldquo;task and process\u0026rdquo; method for evaluators to use when conducting the Mini-CEX. We aim to improve the feasibility and reliability of the Mini-CEX by providing a more structured and transparent approach to assessment.\u003c/p\u003e"},{"header":"Methods","content":"\u003cdiv id=\"Sec3\" class=\"Section2\"\u003e\u003ch2\u003eDesign\u003c/h2\u003e\u003cp\u003eIn a clinical encounter, a resident\u0026rsquo;s task is to collect data for the medical record, then formulate a diagnosis and treatment plan based on that information. During a Mini-CEX, the evaluator may imagine how they would complete the clinical record based on the same encounter\u0026mdash;asking: \u003cem\u003eIs the information gathered sufficient for clinical decision-making?\u003c/em\u003e Thus, the concept of \"task\" becomes the foundation for assessment. The \u0026ldquo;task and process\u0026rdquo; method is derived from this principle.\u003c/p\u003e\u003cp\u003eFor the Mini-CEX\u0026rsquo;s seven domains, specific tasks were identified. First, each domain was assigned a primary task. Then, each task was divided into major and minor components (\u003cem\u003esee\u003c/em\u003e Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The scoring rubric was as follows:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eMajor task completed: Score of 5 (satisfactory)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eMinor task completed: Score of 4 (marginal)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eBoth major and minor tasks completed: Score of 6 (high satisfactory)\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eNeither completed: Score of 3 (unsatisfactory)\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003e Next, the evaluator assessed whether the process matched the ABIM guidelines. Process scores were assigned based on the degree of alignment:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003e\u0026gt;\u0026thinsp;2/3 of guideline behaviors matched: Score of 3\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e1/3\u0026ndash;2/3 matched: Score of 2\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e\u0026lt;\u0026thinsp;1/3 matched: Score of 1\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eThe final score for each item was the sum of the task score and the process score.\u003c/p\u003e\u003cp\u003eFor the item \"Overall Clinical Competence\", the score was calculated differently. Since the process components are already assessed in the other six items, only the task scores were used here. The calculation was based on three core domains\u0026mdash;Medical Interviewing, Physical Examination, and Clinical Judgment:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eIf all three scores\u0026thinsp;\u0026gt;\u0026thinsp;5 \u0026rarr; Overall competence score\u0026thinsp;=\u0026thinsp;9\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eIf two scores\u0026thinsp;\u0026gt;\u0026thinsp;5 \u0026rarr; Score\u0026thinsp;=\u0026thinsp;8\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eIf one score\u0026thinsp;\u0026gt;\u0026thinsp;5 \u0026rarr; Score\u0026thinsp;=\u0026thinsp;7\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eIf all three scores\u0026thinsp;\u0026ge;\u0026thinsp;4 \u0026rarr; Score\u0026thinsp;=\u0026thinsp;6\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eIf any of the three scores\u0026thinsp;\u0026lt;\u0026thinsp;4 \u0026rarr; Use the highest score among the six domains as the overall competence score.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eEthical Considerations\u003c/h3\u003e\n\u003cp\u003e The study protocol was approved by the Review Board of China-Japan Friendship Hospital. Oral informed consent was obtained from all participants.\u003c/p\u003e\n\u003ch3\u003eParticipants\u003c/h3\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e\u003ch2\u003eEvaluators\u003c/h2\u003e\u003cp\u003eSix evaluators (raters) were faculty members from China-Japan Friendship Hospital. All were experienced in conducting and evaluating Mini-CEX and had extensive backgrounds in clinical teaching. The \u0026ldquo;task and process\u0026rdquo; method was co-developed by this group.\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eResidents\u003c/h3\u003e\n\u003cp\u003eSeven residents participated while on clinical rotation. They were informed in advance that five additional evaluators would observe their clinical encounter but would not provide feedback. Their scores would not be recorded in the residents\u0026rsquo; official evaluations. Oral consent was obtained before participation.\u003c/p\u003e\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\u003ch2\u003ePatients\u003c/h2\u003e\u003cp\u003eSeven new patients participated in the Mini-CEX encounters. Informed consent was obtained from each patient before the evaluation.\u003c/p\u003e\u003c/div\u003e\n\u003ch3\u003eStatistics\u003c/h3\u003e\n\u003cp\u003eTo assess inter-rater reliability, intraclass correlation coefficients (ICC) and their 95% confidence intervals were calculated using SPSS version 23 (SPSS Inc., Chicago). A two-way random-effects model with consistency type was used, based on single measurements (k\u0026thinsp;=\u0026thinsp;6 raters). The ICC values were interpreted as follows: \u0026lt; 0.5: Poor reliability; 0.5\u0026ndash;0.75: Moderate reliability; 0.75\u0026ndash;0.90: Good reliability; \u0026gt;0.90: Excellent reliability.\u003c/p\u003e"},{"header":"Results","content":"\u003cdiv id=\"Sec11\" class=\"Section2\"\u003e\u003ch2\u003eParticipants\u003c/h2\u003e\u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e presents information about the study participants. Six evaluators, each from a different department, participated in the study. Between March 2024 and December 2024, seven Mini-CEX encounters were conducted:\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003e2 in the Department of Emergency Medicine\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e2 in the Endocrinology Department\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e1 in the Second Department of Geriatrics\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e1 in the Department of Acupuncture and Moxibustion\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003e1 in the Department of Neurology\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eSeven patients (4 males and 3 females), with an average age of 54\u0026thinsp;\u0026plusmn;\u0026thinsp;8.0 years, were recruited. Their diagnoses included: upper gastrointestinal hemorrhage, severe pneumonia, diabetes mellitus, hypothyroidism, chronic heart failure, neck-angle pain, and Guillain-Barr\u0026eacute; Syndrome (\u003cem\u003esee\u003c/em\u003e Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e\u003c/div\u003e\u003cdiv id=\"Sec12\" class=\"Section2\"\u003e\u003ch2\u003eRatings\u003c/h2\u003e\u003cp\u003eAcross all evaluations, the minimum score was 3, assigned by five evaluators across six different items (excluding overall clinical competence). The maximum score was 9, awarded by Evaluator 5 to Resident 4 in the domains of clinical judgment and overall clinical competence.\u003c/p\u003e\u003cp\u003eIntraclass correlation coefficients (ICCs) were calculated separately for each of the seven Mini-CEX items (\u003cem\u003esee\u003c/em\u003e Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e):\u003c/p\u003e\u003cp\u003e\u003cul\u003e\u003cli\u003e\u003cp\u003eThe lowest ICC was 0.700 for the \u003cem\u003ePhysical Examination\u003c/em\u003e item, followed by \u003cem\u003eCounseling Skills\u003c/em\u003e with an ICC of 0.746, both indicating moderate reliability.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eThe highest ICC was for \u003cem\u003eOverall Clinical Competence\u003c/em\u003e, with a value of 0.904, indicating excellent reliability.\u003c/p\u003e\u003c/li\u003e\u003cli\u003e\u003cp\u003eThe remaining four items had ICCs between 0.75 and 0.90, reflecting good reliability.\u003c/p\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/p\u003e\u003cp\u003eThese results suggest that the \"task and process\" method leads to consistent scoring across raters, particularly for composite or integrative items like overall clinical competence.\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eInformation of the tasks and their breakdowns\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"3\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMajor task\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eMinor task\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eMedical interviewing\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eChief complaint and history of present illness\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003ePast medical history, family history, personal and social history\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePhysical examination\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eChief complaint correlated focused physical examination, find positive signs\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eDifferential negative signs\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eHumanistic qualities/professionalism\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDoctor\u0026rsquo;s perspective: doesn\u0026rsquo;t violate medical practice\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003ePatient\u0026rsquo;s perspective: not awkward,\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eClinical judgment\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDiagnosis, reasonable differential diagnosis\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eFurther test and treatment plan\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCounselling skills\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eExplains rationale for test/treatment\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eObtains a patient\u0026rsquo;s consent\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOrganization/efficiency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eComplete medical interviewing, physical examination and clinical judgement\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eLess than 30 minutes\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOverall clinical competency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eMedical interviewing, physical examination, clinical judgement\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u003cp\u003eHumanistic qualities/professionalism, counselling skills, organization/efficiency\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eInformation of the participants in this study\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"10\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colspan=\"2\" nameend=\"c2\" namest=\"c1\"\u003e\u003cp\u003eEvaluator\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"2\" nameend=\"c5\" namest=\"c4\"\u003e\u003cp\u003eResidents\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colspan=\"4\" nameend=\"c10\" namest=\"c7\"\u003e\u003cp\u003ePatients\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eGender\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eDepartment\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eGender\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003ePGY\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003egender\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003eYear old\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eDepartment\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eDiagnosis\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003ePediatric\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e64\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eEndocrine\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eDiabetes mellitus\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eThe 2nd Geriatric\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e54\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eEmergency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eUpper gastrointestinal hemorrhage\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eEmergency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e46\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eEndocrine\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eHypothyroidism\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eEndocrine\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e2\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e62\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eThe 2nd Geriatric\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eChronic heart failure\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eAcupuncture and Moxibustion\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e42\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eNeurology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eGuillain-barre syndrome\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u003cp\u003eNeurology\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eF\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e57\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eEmergency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eSevere pneumonia\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c3\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c4\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c5\"\u003e\u003cp\u003e3\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e\u003ctd align=\"left\" colname=\"c7\"\u003e\u003cp\u003eM\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c8\"\u003e\u003cp\u003e56\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c9\"\u003e\u003cp\u003eCTM*\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"left\" colname=\"c10\"\u003e\u003cp\u003eNeck-angle pain\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003ctfoot\u003e\u003ctr\u003e\u003ctd colspan=\"10\"\u003e\u0026bull; CTM: the Department of Chinese Traditional Medicine\u003c/td\u003e\u003c/tr\u003e\u003c/tfoot\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003cp\u003e\u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e\u003ccaption language=\"En\"\u003e\u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e\u003cdiv class=\"CaptionContent\"\u003e\u003cp\u003eIntraclass correlation coefficients\u003c/p\u003e\u003c/div\u003e\u003c/caption\u003e\u003ccolgroup cols=\"4\"\u003e\u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e\u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e\u003cthead\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e\u003cp\u003eIntraclass correlation\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colspan=\"2\" nameend=\"c4\" namest=\"c3\"\u003e\u003cp\u003e95% confidence interval\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003ctr\u003e\u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e\u003cth align=\"left\" colname=\"c3\"\u003e\u003cp\u003eLower bound\u003c/p\u003e\u003c/th\u003e\u003cth align=\"left\" colname=\"c4\"\u003e\u003cp\u003eUpper bound\u003c/p\u003e\u003c/th\u003e\u003c/tr\u003e\u003c/thead\u003e\u003ctbody\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eInterviewing\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.866\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.682\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.70\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003ePhysical examination\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.700\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.42\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.926\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eHumanistic qualities/Professionalism\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.876\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.707\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.974\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eClinical judgement\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.86\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.670\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.970\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eCounseling\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.746\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.478\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.940\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOrganization and efficiency\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.856\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.664\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.970\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003ctr\u003e\u003ctd align=\"left\" colname=\"c1\"\u003e\u003cp\u003eOverall clinical competence\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e\u003cp\u003e0.904\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e\u003cp\u003e0.760\u003c/p\u003e\u003c/td\u003e\u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e\u003cp\u003e0.98\u003c/p\u003e\u003c/td\u003e\u003c/tr\u003e\u003c/tbody\u003e\u003c/colgroup\u003e\u003c/table\u003e\u003c/div\u003e\u003c/p\u003e\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eIn medical education, producing a clinical record is a vital component of developing a resident\u0026rsquo;s clinical competence. A clear, concise, and comprehensive record serves as the foundation for patient assessment and treatment planning. After a patient encounter, residents are expected to document their findings and generate a clinical record. In this study, we incorporated the concept of the clinical record into the Mini-CEX, viewing it as a critical task in assessing resident performance.\u003c/p\u003e\u003cp\u003eWe introduced the \"task and process\" method as a dual-step approach to rating:\u003c/p\u003e\u003cp\u003e\u003col\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003eTask completion is used as the first indicator of performance\u0026mdash;determining whether the resident completed the expected major or minor clinical task. This step corresponds to the ABIM's first recommended step of determining whether performance is satisfactory, unsatisfactory, or superior.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003cspan\u003e\u003cli\u003e\u003cp\u003e Process evaluation assesses how closely the resident\u0026rsquo;s behavior aligns with ABIM\u0026rsquo;s guidelines, contributing a process score. This aligns with ABIM\u0026rsquo;s second step of refining the rating within the selected performance category.\u003c/p\u003e\u003c/li\u003e\u003c/span\u003e\u003c/ol\u003e\u003c/p\u003e\u003cp\u003eBy combining task and process scores, a valid and structured final score is achieved. The \u0026ldquo;task and process\u0026rdquo; method acts as a bridge between observation and evaluation, while also linking training to demonstrated competency. Additionally, breaking tasks down into major and minor components helps evaluators apply the Mini-CEX more accurately and systematically.\u003c/p\u003e\u003cp\u003eOur results show that the \"task and process\" method demonstrates moderate to excellent inter-rater reliability. Among all items, \u003cem\u003ePhysical Examination\u003c/em\u003e showed the lowest reliability (ICC\u0026thinsp;=\u0026thinsp;0.700, 95% CI: 0.42\u0026ndash;0.926), followed by \u003cem\u003eCounseling Skills\u003c/em\u003e (ICC\u0026thinsp;=\u0026thinsp;0.746, 95% CI: 0.478\u0026ndash;0.940), both indicating moderate reliability. This may be due to the vague or subjective nature of the task and process definitions for these domains. For example, the physical examination task requires three subjective judgments: identifying the chief complaint, distinguishing major and minor tasks, and comparing observed behavior with guidelines. This complexity introduces variability in scoring.\u003c/p\u003e\u003cp\u003eIn contrast, the \u003cem\u003eOverall Clinical Competence\u003c/em\u003e item showed the highest reliability (ICC\u0026thinsp;=\u0026thinsp;0.904, 95% CI: 0.76\u0026ndash;0.98), likely due to the simplicity of its rating rules. The remaining items showed good reliability, with ICCs ranging from 0.856 to 0.876 (95% CI: 0.664\u0026ndash;0.974). These results suggest that the \"task and process\" method simplifies the rating process while maintaining consistency and clarity for evaluators.\u003c/p\u003e\u003cdiv id=\"Sec14\" class=\"Section2\"\u003e\u003ch2\u003eLimitations\u003c/h2\u003e\u003cp\u003eThis study has several limitations. \u003cem\u003eFirst\u003c/em\u003e, the \u0026ldquo;task and process\u0026rdquo; method is a novel concept introduced by a small group of experienced clinicians. While they are highly familiar with the Mini-CEX, the method may have flaws and requires further refinement. It has only been tested in a single institution, and its reliability and generalizability need to be validated in broader settings. \u003cem\u003eSecond\u003c/em\u003e, the method may restrict evaluator flexibility during both rating and feedback, which could limit nuanced assessments based on individual encounters. \u003cem\u003eThird\u003c/em\u003e, for the domain of \u003cem\u003eHumanistic Qualities/Professionalism\u003c/em\u003e, the minor task involves assessing whether the resident considers the patient\u0026rsquo;s perspective. This poses a challenge, as evaluators are typically trained to view encounters from a physician\u0026rsquo;s standpoint. Shifting to a patient\u0026rsquo;s perspective during rating may be difficult and could impact reliability. These limitations should be addressed in future studies to further improve the method.\u003c/p\u003e\u003c/div\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe \u0026ldquo;task and process\u0026rdquo; method offers a simplified and structured approach for evaluators conducting the Mini-CEX. It provides a clear framework for assessing resident-patient encounters and helps bridge the gap between observation and rating. The method demonstrated moderate to excellent inter-rater reliability, suggesting its potential value in clinical training. However, further research is needed to optimize and validate the approach across different settings and specialties.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e: The authors express their sincere gratitude to the patients who participated in this study.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions:\u0026nbsp;\u003c/strong\u003eQW, JW, XW, NL, KW performed data collection and analysis. All authors contributed to the study\u0026rsquo;s conceptualization, result interpretation, and manuscript draft.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e: none.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u0026nbsp;\u003c/strong\u003eThe dataset supporting the conclusions of this article is included within the article, and is available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthical approval and consent to participate\u003c/strong\u003e: All clinical procedures and study protocols were approved by the Ethics Committee of the China-Japan Friendship Hospital. Written informed consent was obtained from all patients prior to the start of the study.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003e Not applicable.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u003c/strong\u003e The authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eClinical trial number: not applicable\u003c/strong\u003e\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eBerendonk C, Rogausch A, Gemperli A, Himmel W. (2018) Variability and dimensionality of students\u0026apos; and supervisors\u0026apos; mini-CEX scores in undergraduate medical clerkships - a multilevel factor analysis. BMC Med Educ. 2018 May 8;18(1):100. doi: 10.1186/s12909-018-1207-1. PMID: 29739387; PMCID: PMC5941409.\u003c/li\u003e\n\u003cli\u003eGingerich A, van der Vleuten CP, Eva KW, Regehr G. (2014) More consensus than idiosyncrasy: Categorizing social judgments to examine variability in Mini-CEX ratings. Acad Med. 2014 Nov;89(11):1510-9. doi: 10.1097/ACM.0000000000000486. PMID: 25250753. \u003c/li\u003e\n\u003cli\u003eGittinger FP, Lemos M, Neumann JL, F\u0026ouml;rster J, Dohmen D, Berke B, Olmeo A, Lucas G, Jonas SM. (2022) Interrater reliability in the assessment of physiotherapy students. BMC Med Educ. 2022 Mar 16;22(1):186. doi: 10.1186/s12909-022-03231-y. PMID: 35296313; PMCID: PMC8928589. \u003c/li\u003e\n\u003cli\u003eKogan JR, Conforti L, Bernabeo E, Iobst W, Holmboe E. (2011) Opening the black box of clinical skills assessment via observation: a conceptual model. Med Educ. 2011 Oct;45(10):1048-60. doi: 10.1111/j.1365-2923.2011.04025.x. PMID: 21916943. \u003c/li\u003e\n\u003cli\u003eLee V, Brain K, Martin J. (2017) Factors Influencing Mini-CEX Rater Judgments and Their Practical Implications: A Systematic Literature Review. Acad Med. 2017 Jun;92(6):880-887. doi: 10.1097/ACM.0000000000001537. PMID: 28030422. \u003c/li\u003e\n\u003cli\u003eNorcini JJ, Blank LL, Duffy FD, Fortna GS. The mini-CEX: a method for assessing clinical skills. Ann Intern Med. 2003 Mar 18;138(6):476-81. doi: 10.7326/0003-4819-138-6-200303180-00012. PMID: 12639081. \u003c/li\u003e\n\u003cli\u003eRogausch A, Beyeler C, Montagne S, Jucker-Kupper P, Berendonk C, Huwendiek S, Gemperli A, Himmel W. (2015) The influence of students\u0026apos; prior clinical skills and context characteristics on mini-CEX scores in clerkships--a multilevel analysis. BMC Med Educ. 2015 Nov 25;15:208. doi: 10.1186/s12909-015-0490-3. PMID: 26608836; PMCID: PMC4658793. \u003c/li\u003e\n\u003cli\u003eSharma R, Gupta T, Haidery TH, Sinha S, Kumar A. (2022) Current Trends in Mini-Clinical Evaluation Exercise in Medical Education: A Bibliometric Analysis. Cureus. 2022 Dec 30;14(12):e33121. doi: 10.7759/cureus.33121. PMID: 36721606; PMCID: PMC9884390. \u003c/li\u003e\n\u003cli\u003eZaki HA, Yigit Y, Shaban E, Shaban AE, Elmoheen A, Bashir K, Basharat K, Ali M, Hamdi Alkahlout B. (2023) The Utility of the Mini-Clinical Evaluation Exercise (Mini-CEX) in the Emergency Department: A Systematic Review and Meta-Analysis Evaluating the Readability, Feasibility, and Acceptability of Mini-CEX Utilization. Cureus. 2023 Aug 31;15(8):e44443. doi: 10.7759/cureus.44443. PMID: 37791168; PMCID: PMC10544045.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Mini-CEX, Rater idiosyncrasy, Inter-rater variation, Task and process method, Intraclass correlation coefficient","lastPublishedDoi":"10.21203/rs.3.rs-7412906/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7412906/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003ePurpose\u003c/h2\u003e\u003cp\u003eAt present, rater idiosyncrasy in the Mini-CEX remains an issue that needs to be addressed. This study proposes a \"task and process\" method to reduce inter-rater variation and simplify the rating process.\u003c/p\u003e\u003ch2\u003eMethods\u003c/h2\u003e\u003cp\u003eThe \"task and process\" method consists of two steps. While conducting a Mini-CEX, the rater first determines the task associated with each item and whether the resident completes the task. The task is then broken down into major and minor components. Based on how the task is completed, a task score is assigned. Second, the rater observes whether the resident's behavior aligns with the ABIM's Mini-CEX guidelines and assigns a process score. The task score and process score are combined to generate a final score for each item. The inter-rater intraclass correlation coefficient (ICC) was calculated for each item.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e\u003cp\u003eIn 2024, six physicians rated seven residents across seven Mini-CEX encounters. ICCs were calculated for each item, ranging from 0.700 to 0.904, indicating moderate to excellent reliability.\u003c/p\u003e\u003ch2\u003eConclusions\u003c/h2\u003e\u003cp\u003eThe \"task and process\" method offers a structured framework for raters in the Mini-CEX, helping bridge the gap between observation and evaluation. Further studies are needed to refine and validate this approach.\u003c/p\u003e","manuscriptTitle":"Bridging the Gap Between Observation and Rating in the Mini-CEX: Exploring a Task- and-Process Approach","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-07 16:36:34","doi":"10.21203/rs.3.rs-7412906/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"reviewersInvited","content":"","date":"2025-09-25T10:41:11+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-09-03T12:20:04+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-09-01T07:22:58+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-09-01T07:21:35+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2025-08-20T03:19:41+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"17dfa96a-831e-4a16-807e-73f3084479e0","owner":[],"postedDate":"October 7th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-10-07T16:36:34+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-07 16:36:34","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7412906","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7412906","identity":"rs-7412906","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}