Utility of Case-Based Discussions as Workplace Based Assessments in Medical Training: A Scoping Review

Utility of Case-Based Discussions as Workplace Based Assessments in Medical Training: A Scoping Review | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Utility of Case-Based Discussions as Workplace Based Assessments in Medical Training: A Scoping Review Hannah Wu, Karan Varshney, Lawrie McArthur, Nigel Stocks, Lucie Walters This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-8083958/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 7 You are reading this latest preprint version Abstract Background Case-based discussions (CbDs) are a commonly used workplace-based assessment (WBA) method in postgraduate medical education. It is unclear whether they are appropriate for use with medical students, particularly early in their clinical placement experience. The objective of this scoping review is to understand the extent and type of evidence in relation to the utility of CbDs as structured WBA throughout medical training. Methods Joanna Briggs Scoping Review Methodology was used to examine original research articles pertaining to: (population) medical students, doctors, and supervisors; (concept) formal discussion between a single learner and a supervisor about a real patient and (outcome) educational impact, feasibility, acceptability, cost-effectiveness, reliability, and validity. Outcome measures were informed by van der Vleuten and Schuwirth’s Assessment Utility Model. Results There were 4375 papers found of which 131 assessed by full-text analysis. Of the 41 original research studies and 3 systematic reviews included in the analysis. Only one included medical students. Validity of CbD was variable and highest in some studies using assessment tools with behavioural anchors. Protected time and engaged assessors improved validity. There was moderate evidence of change in attitudes and perceptions of learners. Some evidence demonstrated improved clinical reasoning and suggested improvements in patient safety. Learner acceptability was improved by engaged assessor/trainers who provided meaningful, actionable feedback and follow-up. Assessor engagement was reportedly improved with training and in studies involving longitudinal relationships with learners as found in general practice. There were insufficient studies in feasibility and cost-effectiveness, although the time required to undertake CbD was recognised as an important factor affecting both. Conclusion Whilst there was evidence of positive educational impact for CbD, there was a striking scarcity of literature investigating CbDs in medical student learners, which is discordant with the increasing emphasis on workplace-based learning in the contemporary medical education context. Case-based discussion chart stimulated recall medical training medicine workplace-based assessment Figures Figure 1 Introduction Work integrated learning is the foundation of medical education, at all levels of training, where learners develop their expertise working alongside more senior clinicians while making authentic contributions to patient care. Case-based discussions (CbDs) are workplace-based assessment (WBA) tools that allow assessors to explore a trainee’s thinking and clinical reasoning [ 1 ]. The term ‘CbD’ is poly-semantically ambiguous. It can include chart stimulated recall used by the American Board of Emergency Medicine as early as the 1980s [ 2 , 3 ], and random case analysis (RCA) where the case is usually chosen by the assessor. The term is also commonly used to describe various clinical and non-assessment contexts in medical education such as group discussion about a case scenario. The lack of definition and clarity of terminology is a source of confusion for students, educators, and researchers. For the purposes of this scoping review, CbD is defined as “ A structured workplace-based assessment interaction between a learner and supervisor/educator, based on a real patient case .” Assessment Utility The utility of an assessment, such as CbDs, can be considered through van der Vleuten and Schuwirth’s Assessment Utility Model which recognises competing attributes of assessment utility including: validity (including reliability), educational impact, acceptability, practicability and cost-effectiveness [ 4 ]. Several recent systematic reviews have considered the utility of WBAs including CbD in medical education. A 2015 systematic review of the utility of CbD, which specifically excluded studies where CbD was used in combination with other assessment methods, concluded that there was a lack of compelling evidence for the use of patient’s charts to evaluate medical trainees [ 5 ]. In a subsequent systematic review of the usefulness of WBAs in surgical training, only two out of 23 studies focussed on CbD [ 6 ]. These reviews assume all CbD assessment tools have similar qualities and do not differentiate between learners at different levels of expertise such as (novice) early clinical medical student, senior medical student, prevocational trainee and specialist vocational trainee, and (mastery) qualified specialists. The aim of this scoping review is to address this lack of nuance and examine different CbD tools and their utility as WBAs throughout each stage of medical training. Methods The scoping review was conducted in accordance with the JBI methodology for scoping reviews [ 7 ], and in line with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) [ 8 – 10 ]. The protocol was registered with Open Science Framework. Review registration number: https://doi.org/10.17605/OSF.IO/JD86Y Articles were included where they reported on CbDs, including chart stimulated recall (CSR), and random case analysis (RCA), used throughout medical training, from undergraduate medicine to postgraduate specialty training. Eligible participants included: medical students, doctors who are either trainees (prevocational or vocational) or qualified specialists, where these learners were assessed by doctors. Exclusions include: discussion of case scenarios used as a group educational activity or where the learner was not involved in the patient’s care. Original research articles and literature review articles published in English were included in the study. There were no geographical location or publication date restrictions. Search strategies were developed for the PubMed, Scopus (Elsevier), Embase (Ovid), Web of Science (Clarivate), Cumulated Index to Nursing and Allied Health (EBSCO), Education Resources Information Center (ProQuest), PsycInfo (Ovid), and Cochrane Central Register of Controlled Trials databases (Table 1 ). The literature search was undertaken in March 2024. The reference lists of all included sources of evidence were screened for additional studies. Table 1 search strategy Search Query Records retrieved 1 Case-based discussion*[tw] OR Chart stimulated recall*[tw] OR Random case analysis[tw] OR Case discussion*[tw] 2,093 2 General practi*[tw] OR Primary care [tw] OR Medicine [tw] OR Physician*[tw] OR Trainee* [tw] OR Clinician* [tw] OR Medical student*[tw] OR Medical education [tw] OR "Medicine"[mh] OR "Physicians"[mh] OR "Students, medical"[mh] OR "Educational measurement "[mh] OR "Formative feedback"[mh] OR "Education, medical"[mh] OR "Clinical decision making"[mh] 2,965,756 1 + 2 (Case-based discussion*[tw] OR Chart stimulated recall*[tw] OR Random case analysis[tw] OR Case discussion*[tw]) AND (General practi*[tw] OR Primary care [tw] OR Medicine [tw] OR Physician*[tw] OR Trainee* [tw] OR Clinician* [tw] OR Medical student*[tw] OR Medical education [tw] OR "Medicine"[mh] OR "Physicians"[mh] OR "Students, medical"[mh] OR "Educational measurement "[mh] OR "Formative feedback"[mh] OR "Education, medical"[mh] OR "Clinical decision making"[mh]) 1,149 Following the search, all identified citations were collated and uploaded into Covidence software [ 11 ] and duplicates removed. Following a pilot test, titles and abstracts were screened by two independent reviewers to assess the inclusion criteria for the review. Potentially relevant papers were retrieved in full and independently assessed against the inclusion criteria by two reviewers. An excel spreadsheet was created to organise information extracted from articles. One line was allocated to each article with columns relating to specific attributes: the participants, context, assessment tool, study methods, and key findings related the factors in the utility framework [ 12 ]. Messick’s unified validity framework was used to evaluate the validity of the assessment methods [ 13 ]. Data was extracted from articles by two independent reviewers and then compared. Any disagreements in article selection or data extraction were resolved through discussion or with an additional reviewer. Results A total of 4357 articles were screened, 131 were subject to full-text review. Finally, 43 articles were determined eligible for inclusion in this scoping review as outlined in Fig. 1 PRISMA-ScR flow diagram [ 14 ]. Figure 1 . Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram. Study characteristics Of the 43 articles, 18 were set in the UK, five in United States of America, four in Australia, one across Australia and New Zealand combined, four in Canada, two in Pakistan, two in India, one each in Ireland, Europe, Turkey, and Malaysia. Sixteen articles were published between 2015 and 2019. There were 3 systematic reviews [ 5 , 6 , 15 ], and 6 qualitative studies. Quantitative studies included: 7 prospective cohort, 7 cross-sectional and 10 retrospective cohort studies. Mixed methods studies included 1 prospective cohort and 9 cross-sectional studies. The clinical disciplines for the original research were predominantly general practice (13 studies). Table 2 Characteristics of reviews Number of papers Papers REGION AND COUNTRY OF STUDY Europe United Kingdom 18 Aryal 2020, Awad 2015, Bodgener 2011, Bodgener 2017, Brittlebank 2013, Crossley 2011, Eardley 2013, Evans 2005, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Mitchell 2011, Mohanaruban 2017, Mynors-Wallis 2011, Nicol 2018, Phillips 2015, Phillips 2016, Shaw 2014 Ireland 1 Barrett 2016 Europe 1 Primhak 2020 North America United States 5 Holt 2017, Mutter 2022, Rakosfky 2023, Reddy 2018, Solomon 1990 Canada 4 Goulet 2007, Jennett 1995, Nasmith 1995, Norman 1993 Australasia Australia 4 Ingham 2015, Ingham 2016, Morgan 2015, Nair 2012 Australia and New Zealand 1 Castanelli 2019 Malaysia 1 Tahir 2022 Asia India 2 Begum 2023, Shrivastava 2019 Pakistan 2 Jafri 2020, Nadeem 2017 Turkey 1 Yildirir 2023 Multiple 3 Al-Wassia 2015*, Aryal 2021*, Miller 2012* PUBLICATION DATE < 2010 6 Evans 2005, Goulet 2007, Jennett 1995, Nasmith 1995, Norman 1993, Solomon 1990 2010–2014 12 Bodgener 2011, Brittlebank 2013, Crossley 2011, Eardley 2013, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Miller 2010*, Mitchell 2011, Mynors-Wallis 2011, Nair 2012, Shaw 2014 2015–2019 16 Al-Wassia 2015*, Awad 2015, Barrett 2016, Bodgener 2017, Castanelli 2019, Holt 2017, Ingham 2015, Ingham 2016, Mohanaruban 2017, Morgan 2015, Nadeem 2017, Nicol 2018, Phillips 2015, Phillips 2016, Reddy 2018, Shrivastava 2019 2020–2024 9 Aryal 2020, Aryal 2021*, Begum 2023, Jafri 2020, Mutter 2022, Primhak 2020, Rakosfky 2023, Tahir 2022, Yildirir 2023 LEVEL OF LEARNER Medical student Preclinical years 0 clinical years 1 Tahir 2022 Prevocational 2 \ Mitchell 2011, Nair 2012 Vocational training 31 Aryal 2020, Awad 2015, Bodgener 2011, Bodgener 2017, Brittlebank 2013, Castanelli 2019, Eardley 2013, Evans 2005, Goulet 2007, Holt 2017, Ingham 2015, Ingham 2016, Jafri 2020, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Morgan 2015, Mutter 2022, Nadeem 2017, Nasmith 1995, Nicol 2018, Phillips 2016, Primhak 2020, Rakosfky 2023, Reddy 2018, Shaw 2014, Yildirir 2023, Consultant level 5 Jennett 1995, Mynors-Wallis 2011, Norman 1993, Phillips 2015, Solomon 1990 Multiple levels 4 Al-Wassia 2015*, Aryal 2021*, Miller 2012*, Mohanaruban 2017 ASSESSMENT TOOL STYLE Behavioural anchors 8 Awad 2015, Castanelli 2019, Crossley 2011, Evans 2005, Goulet 2007, Norman 1993, Reddy 2018, Shrivastava 2019 Feedback prompts 5 Bodgener 2017, Ingham 2015, Ingham 2016, Jennett 1995, Rakosfky 2023 Normative comparisons 9 Eardley 2013, Jafri 2020, Johnson 2011, Jyothirmayi 2012, Mitchell 2011, Mynors-Wallis 2011, Nadeem 2017, Solomon 1990, Tahir 2022, Not described 21 Al-Wassia 2015*, Aryal 2020, Aryal 2021*, Barrett 2016, Begum 2023, Bodgener 2011, Brittlebank 2013, Holt 2017, Mehta 2013, Miller 2010*, Mohanaruban 2017, Morgan 2015, Mutter 2022, Nair 2012, Nasmith 1995, Nicol 2018, Phillips 2015, Phillips 2016, Primhak 2020, Shaw 2014, Yildirir 2023 DISCIPLINE OF STUDY Single discipline studies General practice / community medicine 13 Bodgener 2011, Bodgener 2017, Evans 2005, Goulet 2007, Ingham 2015, Ingham 2016, Jennett 1995, Morgan 2015, Nasmith 1995, Nicol 2018, Norman 1993, Shaw 2014, Shrivastava 2019 Medical disciplines 10 Barrett 2016, Crossley 2011, Holt 2017, Johnson 2011, Jyothirmayi 2012, Mohanaruban 2017, Mutter 2022, Primhak 2020, Reddy 2018, Yildirir 2023 Surgical disciplines 5 Aryal 2020, Awad 2015, Eardley 2013, Phillips 2015, Phillips 2016 Psychiatry 3 Brittlebank 2013, Mynors-Wallis 2011, Rakosfky 2023 Anaesthetics 1 Castanelli 2019 Chemical pathology 1 Jafri 2020 Emergency medicine 1 Solomon 1990 Obstetrics and Gynae 1 Tahir 2022 Paediatrics 1 Mehta 2013 Radiology 1 Nadeem 2017 Not discipline specific 6 Al-Wassia 2015*, Aryal 2021*, Begum 2023, Miller 2010*, Mitchell 2011, Nair 2012 STUDY DESIGN Literature review 3 Al-Wassia 2015, Aryal 2021, Miller 2010 Qualitative studies 6 Ingham 2015, Ingham 2016, Jennett 1995, Jyothirmayi 2012, Nicol 2018, Phillips 2016 Quantitative Prospective cohort 7 Brittlebank 2013, Mynors-Wallis 2011, Nadeem 2017, Nasmith 1995, Nicol 2018, Norman 1993, Reddy 2018 Cross-sectional 7 Begum 2023, Evans 2005, Johnson 2011, Morgan 2015, Phillips 2015, Tahir 2022, Yildirir 2023 Retrospective cohort 10 Awad 2015, Barrett 2016, Bodgener 2017, Castanelli 2019, Crossley 2011, Eardley 2013, Goulet 2007, Mitchell 2011, Shaw 2014, Solomon 1990, Mixed methods Prospective cohort 1 Nair 2012 Cross-sectional 9 Aryal 2020, Bodgener 2011, Holt 2017, Jafri 2020, Mehta 2013, Mohanaruban 2017, Mutter 2022, Primhak 2020, Shrivastava 2019 *Review articles Assessment tools Eight original research studies used behavioural anchors,[ 16 – 23 ] while five studies focussed on providing feedback prompts to supervisors. [ 24 – 28 ] Nine studies used tools with normative comparisons such as below, at or above expected standard for the level of trainee.[ 29 – 37 ] The majority of studies did not specifically describe the assessment tool. Learner, trainers and disciplines CbD within vocational training was described in 31 articles [ 16 , 17 , 19 , 20 , 22 , 24 – 26 , 28 – 32 , 35 , 38 – 49 ]. No articles described CbD undertaken by transition-to-clinical (novice) medical students, one studied clinical medical students [ 37 ], two involved prevocational trainees such as foundation year or intern,[ 33 , 50 ] one included both pre-vocational and vocational trainees [ 51 ], and five involved fully qualified clinician .[ 21 , 27 , 34 , 36 , 52 ]. Utility The review found 23 original research studies reporting on validity.[ 16 – 18 , 20 – 27 , 29 , 31 , 33 – 39 , 41 , 45 , 53 ] Another 17 reported on educational impact,[ 5 , 17 , 25 , 26 , 28 , 31 – 33 , 38 – 43 , 51 , 52 , 54 ] 18 papers described practicability,[ 17 , 19 , 20 , 25 , 26 , 30 – 32 , 34 , 38 – 40 , 47 , 50 – 54 ] seven papers described acceptability.[ 25 , 38 – 40 , 46 , 47 , 50 ] Four studies described cost but only hinted at cost-effectiveness.[ 20 , 21 , 24 , 50 ] Validity Face validity describes the subjective assessment of whether CbD appears to measure the constructs it is intended to measure. Table 3 illustrates that when assessment tools used feedback prompts or normative comparisons, CbDs were seen as a useful component of assessment of competence, offering greater capacity to unpack thinking than consultation observations [ 26 , 34 ]. Face validity was reportedly improved with recency of consultation, an appropriate setting with adequate time for assessment, and assessor training [ 26 , 38 , 41 ]. In one study where the assessment tool was not described, paediatric trainees described assessments as more beneficial in discussions about more challenging cases [ 41 ]. Only one study assessing face validity described an assessment tool with behaviourally anchored descriptors, however face validity of this tool was only described in terms of expert opinions of the domains and scale descriptors [ 23 ]. Table 3 Validity study findings Description of CbD tool Author/ Setting Learners Description of validity STUDIES OF FACE VALIDITY Behaviourally anchored scale CbD tool had five domains related to a specific EPA marked against behaviourally anchored descriptors (1–10). Shrivastava 2019 India Community medicine 3 community medicine trainees Copies of the assessment tools were sent to seven subject experts, who universally agreed that CbD tool descriptors and scale content validity matched expected EPAs. Feedback prompts Formative random chart assessment tool with prompts for feedback against domains including: history taking, examination, clinical reasoning and management etc. Ingham 2016 Australia General practice 10 GP trainees and medical educators The two formative assessment approaches (RCA and direct observation) provide complementary insights. RCA of less value when consultations were not recent, as registrars could not recall clinical reasoning. Ingham 2015 Australia General practice 10 GP trainees, and 8 medical educators RCA reportedly enabled overview of complexity, scope of practice and types of cases managed. Increased medical educators’ confidence in discriminating high performers. The ability to look back through the notes in RCA enabled educators to uncover how a registrar managed a patient when not observed and over a number of consultations. Semi-structured interview addressing: problem identification, history, physical, treatment, and follow-up. Jennett 1995 Canada Family medicine 20 family physicians Chart-stimulated recall measured clinical reasoning better than chart audit alone. Normative comparison Eight domains of good psychiatry practice. Likert scale 0–5 (fails to meet standards to excels). Mynors-Wallis 2011 UK Psychiatry 86 psychiatry consultants 87% thought that case-based discussion could be a useful component for revalidation. Average scores over the eight domains increase with the number of attempts. Assessment tools not described. Mehta 2013 UK Paediatrics 26 paediatric trainees Trainees felt challenging cases resulted in more beneficial feedback (through reflective practice). To ensure validity trainees reported assessor commitment, training, appropriate setting and adequate time needed. Bodgener 2011 UK General practice 45 GP trainees, 30 GP trainers Assessment was of value in demonstrating thinking and partially assessed performance. Structured assessment could focus on specific issues. Consultation observations considered a better as CbD could be gamed with adequate preparation and background reading. STUDIES OF CONSTRUCT VALIDITY Behavioural anchors Form consists of eight items. Each item is rated as either D: development required, S: satisfactory, O: outstanding or N: not assessed. Awad 2015 UK Surgery (ENT) 46 surgical trainees 1456 Case-based discussions Grades were significantly different between core training 88.7% (CI: 87.4–89.9%), and specialist training 95.3% (CI: 94.7–95.9%). CbDs were highly sensitive for competence (99%), yet not specific. Assessors received training. Charts were assessed for quality of record keeping and quality of care based on explicit criteria. Marked as satisfactory = 1, unsatisfactory or NA = 0 Goulet 2007 Canada Family medicine 25 family physicians, 16 to 48 charts were assessed for each. Concordance between chart review alone and that of chart review with CSR interview was 75% for chart keeping, 69% for clinical investigation, 81% for diagnostic accuracy, and 74% for treatment plan.) Rating forms with seven-point scales and checklists to rate and score each domain: eg data acquisition, problem solving, management. Norman 1993 Canada General practice Ontario primary care physicians CSR discriminated among subgroups of clinicians: 26 randomly selected reference physicians, 20 self-referred and 37 referred by the licensing body. Correlation between CSR and other assessments eg, 0.48 with MCQs, 0.78 with standard patients, 0.76 with oral test, 0.38 with OSCEs. Normative comparison Structured grading sheet – 4 case-discussion items each marked using a six- point Likert scale (very poor to excellent). Tahir 2022 Malaysia Obstetrics 80 medical students Student self-assessment vs faculty assessment of case discussion) correlation coefficient of 0.307 (p < 0.01) with students underrating their performance. CbD tool had seven items each marked using marked using a six-point Likert scale (below expectations to Above expectations for F1) Mitchell 2011 UK Foundation year 1646 Foundation year doctors Odds ratios for association between mean WBA scores and training difficulties. For CbD (OR = 0.39; p = 0.002). A weak predictive value for diagnosing ‘in difficulty’ status (0.64). Six competencies assessed eg clinical management) assessed using 6-point Likert scale for standard for stage of training (3 = borderline) Eardley 2013 UK Surgery 2736 trainees completed 17904 CbD Only 0.55% of scores below 4/6 No relation between stage of training and summary rating given by assessor. Means – 6.5 assessments per trainee, 20minutes assessment, 9minutes feedback. Domains included data acquisition, problem solving, patient management, resource utilisation etc rated on an 8-point scale ( > = 5 = acceptable). Solomon 1990 US Emergency medicine 8 ED consultants Correlation between CSR and simulated patient encounters was 0.49 (p < 0.01) Correlation between recertification and oral certification (0.37, p < 0.005) and written certification (0.45, p < 0.05) scores. Assessment tools not described. Barrett 2016 Ireland Medicine 261 CbDs Low number of assessments documenting borderline or below expectation competence. Expectations system did not explicitly reference curriculum outcomes or competencies. Shaw 2014 UK GP 116 GP trainees CbD scores were not statistically sig different between cohorts passing and failing hurdle clinical skills assessment however higher numbers of CbD was associated with failing the final clinical skills assessment. Brittlebank 2013 UK Psychiatry 220 psychiatry trainees 574 CbD High correlations occurred between CbD, peer assessment, and presentational assessments. STUDIES OF RELIABILITY Behavioural anchors Two 11-item scoring forms were developed with either a Milestone Based Rating Scale (using Internal Medicine Milestones) or Standard Rating. Reddy 2018 USA Medicine Physician trainees Faculty scored With use of Milestone Based Rating Scale inter-rater reliability improved from 0.51 (with standard rating scale) to 0.61 (small effect). Form consists of eight items. Each item is rated as either D: development required, S: satisfactory, O: outstanding or N: not assessed. Awad 2015 UK ENT surgery 46 surgical trainees 1456 CbD CbD was found to be reliable (Cronbach’s a = 0.848). Study compared CbD conventional and construct-aligned scales Crossley 2011 UK Physician 2448 trainees 4513 CbD Using construct-aligned scales reduced CbD required from 8 to 3 for R = > 0.7 Rating forms with seven-point scales and checklists to rate and score each domain Norman 1993 Canada General practice Ontario primary care physicians Test reliability was high, ranging from 0.73 to 0.91, and all tests dis- criminated among subgroups of clinicians. Normative comparison Undisclosed number of items and “overall clinical judgement” were rated from ‘well below expectations to ‘well above expectations” Johnson 2011 UK Medicine specialities 42 physician trainees Total of 12 CbD required for reliability 0.7 No CbD scores were ‘below’ or ‘well below expectations’. Eight domains of good psychiatry practice used as CbD assessment items. Likert scale 0–5 (fails to meet standards to excels). Mynors-Wallis 2011 UK Psychiatry 86 psychiatry consultants Item-total correlations for 8 CbD domains between 0.65–0.86 R = 0.84 The issue of how cases are selected is important. Most doctors in the study supported random selection. Domains included data acquisition, problem solving, patient management, resource utilisation etc rated on an 8-point scale ( > = 5 = acceptable care. Solomon 1990 US Emergency medicine 8 ED consultants Reliability R = 0.54 for 3 CSR cases Inter-rater reliability across 3 CSR cases was 0.70 (p < 0.03). Competences selected by the trainee from course list. Each competency marked as either: insufficient evidence, needs further development, competent or excellent. Bodgener 2017 UK General practice 162 CbD from 30 GP trainees. Significant variance between assessors (hospital staff vs consultants vs GPs), with GPs providing lower scores. Assessment tool not described. Castanelli 2019 ANZ Anaesthetics 11,125 CbD from 1,411 trainees CbD had higher reliability than DOPS and MSF With greater weighting for more reliable assessments, acceptable composite reliability for interim and final decisions is possible with nine (R = 0.7) and 15 WBAs (R = 0.8) Brittlebank 2013 UK Psychiatry 220 psychiatry trainees 574 CbD R = 0.80 after 100 minutes of assessment time gathered over 4 episodes, each with different assessors. CbD = case based discussions, CSR = chart stimulated recall, DOPS = direct observation of procedural skills, EPA = Entrustable professional activities, MCQs = multiple choice questions, MSF = multi-source feedback, OSCEs = observed structured clinical examinations, RCA = random case analysis, WBA = workplace-based assessment. Construct validity measures the extent to which a test covers all aspects of the construct it aims to measure and is often described in teams of correlations to other tests which seek to assess the same learning outcomes. Tools with behaviourally anchored scales showed high construct validity with good discrimination between candidate cohorts, reasonable concordance with chart review alone, standard patients (r = 0.78), and oral tests (0.76) [ 16 , 20 , 21 ]. In another study where the assessment tool was not clearly described, high correlations occurred between case based discussions peer assessments and presentation assessments [ 39 ]. CbD with normative comparison did not fair as well, with poor correlations with student self-assessment, simulated patient encounters, oral and written certification scores [ 36 , 37 ]. There was weak predictive value of diagnosing learners in difficulty and only 0.55% of assessments scoring below 4/6 [ 29 , 33 ]. However, in one study which did not describe assessment tools, increased number of CbDs undertaken by GP trainees was associated with increased likelihood of failing [ 45 ]. Reliability of CbD tool is the likelihood that a candidate will receive the same grade for the same assessment. Overall, reliability was frequently reported in the order of 0.7 (after around 3–9 case based discussions or 100 minutes of assessment) to 0.8 (after 12–15 case based discussions) [ 16 – 18 , 34 , 39 ]. In one study behavioural anchors improved reliability when compared to a normative comparison tool (from 0.51 to 0.61) [ 22 ]. The type of assessor also appeared to influence reliability with one study showing lack of reliability with increasing seniority of hospital-based assessors for GP trainees compared to GP educators [ 24 ]. Educational impact Educational impact was considered from the perspective of feedback received (Kirpatrick level 1), reported changes in knowledge or skills (level 2) and evidence of improvement on further testing (level 3).[ 55 ] Level 4 changes in organisational practice and benefits to patients was not found. Several studies indicated that CbD was a preferred work-based assessment tool by examiners who reported they could gain deep insight, enable rich discussions and provide feedback on the learner’s consulting, specific aspects of patient care and how these could be improved in the future, including development of future learning objectives [ 19 , 28 , 31 , 32 , 38 , 44 , 54 ]. Agility of the tool was important in enabling exploration of the trainee’s clinical reasoning and broadening their higher order thinking [ 25 , 28 , 32 , 42 , 52 ]. CbD can be described as having three phases: reporting, analysing and managing [ 48 ]. Trainer’s skills in providing feedback impacted on the educational value [ 38 , 44 ]. Feedback could be seen as ‘tick a box’ and not specific enough to support learning, particularly for less competent learners where concern was raised that supervisors were ‘afraid of failing’ trainees [ 44 , 51 ]. When learners had many CbD to do they also described wanting to ‘just get them done’ with little regard to the feedback received [ 48 ]. Lack of follow-up by examiners was seen as a missed opportunity to consolidate learnings [ 53 ]. Changes in attitudes and perceptions were described in a few studies, including increasing trainees’ own awareness of influencing factors on treatment decisions,[ 28 ] aiding clinical decision-making and effecting a change in practice [ 41 ]. Some studies purported to encourage reflective learning based on the case and performance in the CbD [ 31 , 38 , 41 , 43 , 44 ]. In one study 23% of medical trainees undertook self-directed learning in response to their CbD, while 20% indicated they did not change their practice [ 51 ]. More senior learners described that more challenging cases resulted in more beneficial feedback [ 41 ]. Evidence of concrete behaviour change was demonstrated by radiology residents who, when compared to peers who did not undertake CbD, improved their reporting skills following weekly CbDs.[ 35 ] A positive impact on resident behaviour was reported following CbD with faculty (facilitators) self-reporting a change in residents’ documentation skills and insight into hospital systems-based issues [ 40 ]. Only two studies described systems level benefit from undertaking an RCA activity [ 42 ]. Morgan found that 29.8% of the Australian GP supervisors surveyed in their study, reported that RCA enabled identification of critical patient safety issues that resulted in a change in patient management, including delayed diagnosis, inappropriate test ordering, inadequate follow-up, and poor quality of clinical records [ 42 ]. A proportion (15.8%) of trainers also described that undertaking the RCA with the trainee resulted in a change to the original management plan, suggesting improved patient care [ 42 ]. Practicability Practicability, including feasibility, can be considered from the perspective of how the assessment is implemented and managed within the clinical environment. Themes relating to practicability within the clinical setting included: (i) implementation, including training of assessors and orientation of learners and (ii) the need for protected time and space, where both learner and assessor can focus. Implementation of CbD assessments in the clinical setting were not well described in the literature. Educators went to considerable effort to develop and refine a specific CbD tool in consultation with supervisors and learners [ 22 , 23 , 28 ]. Several studies described efforts to orientate faculty [ 40 ], improve supervisor feedback [ 19 ] and provide additional training to senior staff completing CbD assessments (from several hours to six months) [ 20 , 42 , 48 ]. Many studies recognised that CbD can be time-consuming to complete, particularly if the assessment rubrics are long [ 31 , 41 , 51 ]. The average time taken to complete a CbD was 20–25 minutes, with the total assessment time in one program designed for international medical graduates being 10 hours [ 30 , 35 , 39 ]. Potential pitfalls in the implementation of CbD primarily related to its time-consuming nature [ 51 ]. Sufficient protected time was necessary for CbDs to be seen as effective [ 32 ]. Trainees with little power in the clinical setting found it difficult to initiate CbDs and complete the required number [ 52 ]. Acceptability Acceptability specifically relates to learner and assessor attitude toward and engagement in the CbD assessment process. Some studies used survey tools and found high measures of satisfaction [ 30 , 39 , 42 , 52 ] and agreement that CbD could be applied in their training context.[ 46 ] Other studies explored acceptance through qualitative interview [ 28 , 32 ]. Despite its time-consuming nature, CbD was frequently seen as highly acceptable as a result of its high educational value, particularly when used as a formative tool [ 28 , 31 , 32 , 38 ]. One study described that formative CbD alleviated trainees stress related to uncertainty [ 47 ]. In one study of surgical trainees, the benefit was felt to be greater for those at a junior stage of training [ 44 ]. Trainee’s attitudes to CbD were influenced by their relationship with the assessor, high levels of engagement and their approach to feedback.[ 25 , 41 , 51 ] Time constraints had a negative impact on feedback [ 41 ]. One suggestion to improve acceptability was to use a combination of ad hoc and pre-planned assessments [ 51 ]. The impact of case selection, including proximity to time of patient consultation, on the perceived benefit of CbD was highlighted in a few studies [ 26 , 41 ]. Cost There were no articles that discussed the overall cost-effectiveness of CbD, although some studies discuss issues pertaining to cost. Costs have been reported more generally for WBA programs. In competency assessment of primary care physicians, the cost of a peer review activity was approximately $ 850 CAN for each peer assessment and feedback session, an annual cost of $ 935 000 CAN in 1993 [ 21 ]. In an Australian postgraduate context, the estimated cost of delivering a whole WBA program (where 2 CbD were included in a total of 22 assessments over 6 months) was estimated to be $ 6000 AUD per international medical candidate in 2012 [ 50 ]. Like all assessment trade-off decisions, the cost must be balanced against the potential improvement in sensitivity in picking up learner problems [ 20 ]. Discussion This scoping review unveiled a high degree of heterogeneity when examining CbD used throughout the entire pipeline of medical training. Validity of CbD as an assessment tool can be improved with carefully designed assessment tools with behavioural anchors, and appropriate setting with protected time. There was evidence for educational impact at low levels of Kirkpatrick’s hierarchy, with some evidence of behavioural change (level 3) and scant evidence of system and patient impact. Learner acceptability is improved by engaged assessor/trainers who provide meaningful actionable feedback and follow-up. Assessor engagement may be improved with training and longitudinal relationships with learners as found in general practice. There were insufficient studies in feasibility, and cost-effectiveness; although the time required to undertake CbD is an important factor. Fit-for-purpose assessment does not require perfect ‘utility’, and it is these trade-offs that educators need to consider with ‘purpose’ at front of mind. The diversity of design of studies in this review may reflect the various contexts in which CbD is adopted, ranging from general practice (GP), medicine and subspecialties, surgery and subspecialties, to psychiatry. It is not surprising that a GP setting was the most common setting in which CbD was researched; this may reflect the inherent 1–1 GP supervisor-trainee relationship in clinical practice. It may also be that CbD aptly caters for the diverse nature and style of consultations in primary care, compared with hospital training structures where supervisors and trainees are less closely connected in the work setting. Validity of CbD faired well compared with other work-based assessments and they are often used as one of a suit of assessments in vocational and prevocational training. Inconsistency in CbD items and marking rubrics creates potential issues for supervisors or examiners who supervise trainees of diverse backgrounds and stages or training. When learning constructs are not clearly defined in the marksheet the validity of the assessment may be threatened. Reliability was overall acceptable (r = 0.7–0.8), but the number of CbDs needed to achieve this varied between studies (3–15). Of note, the inter-rater reliability was impacted by the assessor’s clinical background and level of experience, assessor training and familiarity of standard expected of the trainee. Documenting evidence of validity is essential if the assessment is used for high stakes assessments such as certifying competency. If CbD is intended as assessment for learning, it is more relevant to optimise the educational impact. Most evidence regarding educational impact pertained to levels 1 and 2 of Kirkpatrick’s hierarchy [ 55 ]. Measuring educational value as a reaction can be easily captured, but there are greater practical barriers for researchers when evaluating higher levels of impact such as changes in behaviour, organisational practice, and patient care. Repeating an assessment with the same trainee longitudinally may be feasible in a GP setting, but less feasible in hospital-based systems where the supervisory and assessment structure is more distributed. Practicability, including implementation-related issues was not well researched. Trainee and trainer’s reactions to CbD highlighted that the time-consuming nature of CbD and the need for protected time. This raises potential issues of equity of access when learners require significant numbers of CbD to meet assessment requirements. It also highlights the effort required to engage and maintain engagement of clinical supervisors within the training system. The acceptability of CbDs was frequently high, particularly were feedback contributed to further development of clinical reasoning. While this is appropriate for postgraduate doctors who have developed this skill, medical students are not yet at a mastery stage. Their views on the acceptability of CbD need further exploration. Clinician time is a valuable resource and for a setting such as fee-for-service general practice, direct financial losses are tangible and cannot be disregarded. Context-specific considerations must be made for how the issue of cost-effectiveness can be overcome. In some countries like Australia, GP supervisors can received a payment for clinical placement supervision of learners e.g., $ 200 per half day to have a medical student. However, after accounting for reduced patient billings to enable students to have an active role in patient care, and then ‘blocked off’ time to undertake CbDs, GP supervisors may still face an overall financial loss. GP supervisors reported motivation to teach teaching include: enjoyment, contribution to the profession and community, adding variety, and workforce succession planning, with less motivation stemming from financial reward [ 56 ]. Cost effectiveness for training institutions must account for cost of training clinician supervisors to optimise validity and educational value for learners, potentially tipping the evaluation of effectiveness of a costly investment more positively. While assessor training was consistently reported as a recommendation for improving the experience of CbD, articles stopped short of making specific recommendations for what this encompassed. The main limitations of this articles included in this review is the large number of studies which do not adequately describe the structure and focus of the assessment tool and the context of its use in the CbD,[ 57 ] making it impossible to fully understand utility findings. Further, in this study utility has been assessed through van der Vleuten and Schuwirth’s Assessment Utility Model which underplays the difficulty of making competence explicit and measurable [ 12 , 58 ]. The medical education context is increasingly moving towards longitudinal, community or workplace-based exposure early within medical degrees. The selection of assessment programmes should be informed by directly relevant evidence. This review has highlighted a striking scarcity of literature investigating the utility of CbDs in medical student learners, with only one study examining its use in clinical level medical students [ 37 ], and no studies in transition-to-clinical practice level learners. It is reasonable to consider the effects of WBA in the postgraduate education space in translating this to the undergraduate context. Despite these limitations, this scoping review provides useful learnings for educational designers to optimise the utility of CbD taking into account the purpose and context of CbD in their medical education program. Conclusions This scoping review has mapped the current literature on the utility of case-based discussions as a structured workplace-based assessment. Validity can be improved with carefully designed assessment tools with behavioural anchors, and appropriate setting with protected time for CbD. Educational impact including positive learner reactions and demonstrates some improvement in learner skills. Learner acceptability is improved by engaged assessor/trainers who provide meaningful, actionable feedback. The time required to undertake CbD is an important factor in practicability and acceptability. There was a striking scarcity of literature investigating the utility of CbD in medical student learners, which is discordant with the increasing emphasis on workplace-based learning and assessment in medical schools. Declarations Ethics, and Consent to Participate: not applicable. Consent to publish: The authors each consent to publish. Funding This research project is supported by The Royal Australian College of General Practitioners with funding from the Australian Government under the Australian General Practice Training Program. Author Contribution HW wrote the scoping review protocol and registration, designed the search strategy, screened the articles, extracted data, undertook data analysis, and contributed to the manuscript.KV contributed to the scoping review protocol and registration, screened the articles, extracted data, and contributed to the manuscript.LW undertook data analysis and contributed to the manuscript.NS and LM critically reviewed the manuscript. Acknowledgement This research project is supported by The Royal Australian College of General Practitioners with funding from the Australian Government under the Australian General Practice Training Program.Thank you to Jasmine Castellano, Faculty of Health and Medical Sciences Liaison Librarian, for assistance with developing the search strategy. Data Availability All data supporting the findings of this study are available within the paper and its Supplementary Information. References Schipper S, Ross S. Structured teaching and assessment: a new chart-stimulated recall worksheet for family medicine residents. Can Fam Physician. 2010;56(9):958–9. Maatsch J, et al. Predictive Validity of Medical Specialist Examinations. Final Report for Grant HS 02038-04, National Centre of Health Services Research. East Lansing, MI: Michigan State University; 1983. Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide 31. Medical teacher, 2007. 29(9–10): pp. 855–71. van der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309–17. Al-Wassia H, et al. Using patients' charts to assess medical trainees in the workplace: a systematic review. Med Teach. 2015;37(Suppl 1):S82–7. Aryal K, Hamed M, Currow C. The usefulness of work-based assessments in higher surgical training: a systematic review. Int J Surg. 2021;94:106127. Aromataris E, Porritt LC, Pilla K, Jordan B Z, editors. JBI Manual for Evidence Synthesis. Editor: JBI; 2024. Peters MD, et al. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141–6. Tricco AC, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467–73. Peters MDJ, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synthesis. 2020;18(10):2119–26. Innovation VH. Covidence systematic review software. Australia: Melbourne; 2024. Van Der Vleuten CP. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1996;1(1):41–67. Messick S. Validity.. In: Linn RL, editor Educational measurement., 1989. New York: Macmillan. pp 13–103. Page MJ, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Revista Esp Cardiologia (English Edition). 2021;74(9):790–9. Miller A, Archer J. Impact of workplace based assessment on doctors’ education and performance: a systematic review. BMJ, 2010. 341. Awad Z, et al. Utilisation and outcomes of Case-based discussion in otolaryngology training. Clin Otolaryngol. 2015;40(2):86–92. Castanelli DJ, et al. The reliability of a portfolio of workplace-based assessments in anesthesia training. Can J Anesth - J Canadien D Anesthesie. 2019;66(2):193–200. Crossley J, et al. Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ. 2011;45(6):560–9. Evans A, Ormston B, Dunbar A. Development of a profile to aid feedback on trainers' skills in one-to-one teaching using random case analysis. Educ Prim Care. 2005;16(3):283–94. Goulet F, et al. Assessment of family physicians' performance using patient charts - Interrater reliability and concordance with chart-stimulated recall interview. Evsluation Health Professions. 2007;30(4):376–92. Norman GR, et al. Competency Assessment of Primary Care Physicians as Part of a Peer Review Program. JAMA: J Am Med Association. 1993;270(9):1046–51. Reddy ST, et al. Preliminary Validity Evidence for a Milestones-Based Rating Scale for Chart-Stimulated Recall. J Graduate Med Educ. 2018;10(3):269–75. Shrivastava SR, et al. Development, validation and use of appropriate assessment tools for certification of entrustable professional activities in community medicine to produce a competent postgraduate: A pilot study. Indian J Public Health. 2019;63(4):277–81. Bodgener S, Denney M, Howard J. Consistency and reliability of judgements by assessors of case based discussions in general practice specialty training programmes in the United Kingdom. Educ Prim Care. 2017;28(1):45–9. Ingham G et al. ARCADO - Adding random case analysis to direct observation in workplace-based formative assessment of general practice registrars. BMC Med Educ, 2015. 15. Ingham G, Fry J, Ward B. Adding random case analysis to direct observation (ARCADO) - Updating the external clinical teaching visit to improve general practice registrar assessments. Aus Fam Physician. 2016;45(12):918–20. Jennett PA, et al. Patient charts and physician office management decisions: chart audit and chart stimulated recall. J Continuing Educ Health Professions. 1995;15(1):31–9. Rakofsky JJ, et al. A Chart-Stimulated Recall Activity to Assess Psychiatry Residents' Treatment-Based, Clinical Reasoning Skills. Acad Psychiatry. 2023;47(6):663–6. Eardley I, et al. Workplace-based assessment in surgical training: experiences from the Intercollegiate Surgical Curriculum Programme. ANZ J Surg. 2013;83(6):448–53. Jafri L et al. Fostering teaching-learning through workplace based assessment in postgraduate chemical pathology residency program using virtual learning environment. BMC Med Educ, 2020. 20(1). Johnson G, et al. Assessing trainees in the workplace: results of a pilot study. Clin Med. 2011;11(1):48–53. Jyothirmayi R. Case-based discussion: assessment tool or teaching aid? Clinical Oncology (Royal College Radiology), 2012. 24(10): pp. 649–53. Mitchell C, et al. Workplace-based assessments of junior doctors: do scores predict training difficulties? Med Educ. 2011;45:1190–8. Mynors-Wallis L, et al. Case-based discussion: a useful tool for revalidation. Psychiatry Bull. 2011;35(6):230–4. Nadeem N, et al. Chart-stimullated Recall as a Learniing Tool for Improving Radiology Residents' Reports. Acad Radiol. 2017;24(8):1023–6. Solomon DJ. and An Assessment of an Oral Examination Format for Evaluating Clinical Competence in Emergency Medicine. Acad Med, 1990. 65(9). Tahir ABIM, et al. Validity of Medical Students Self-Assessment of Proficiency in Clinical Long Case Examination. Malaysian J Med Health Sci. 2022;18(5):41–6. Bodgener S, Tavabie A. Is there a value to case-based discussion? Educ Prim Care. 2011;22(4):223–8. Brittlebank A, et al. Workplace-Based Assessments in Psychiatry: Evaluation of a Whole Assessment System. Acad Psychiatry. 2013;37(5):301–7. Holt SR, Sofair AN. Resident and Faculty Perceptions of Chart-Stimulated Recall. South Med J. 2017;110(2):142–6. Mehta F, Brown J, Shaw NJ. Do trainees value feedback in case-based discussion assessments? Med Teach. 2013;35(5):e1166–72. Morgan S, et al. Clinical supervision using random case analysis in general practice training. Educ Prim Care. 2015;26(1):40–6. Nicol J, Cattanach K, Mhaoileoin AN. The use of case-based discussions to develop peer assessment skills among general practice trainees. Educ Prim Care. 2018;29(1):43–5. Phillips A, et al. Case-based discussions: UK surgical trainee perceptions. Clin Teacher. 2016;13(3):207–12. Shaw B, et al. An investigation of factors affecting the outcome of the clinical skills assessment (CSA) in general practice specialty training. Educ Prim Care. 2014;25(2):91–5. Yildirir A, Turan S. Assessment and Evaluation of Cardiology Residency Training in Türkiye: A National Survey. Anatol J Cardiol. 2023;27(10):580–91. Mutter M, et al. Use of chart-stimulated recall to explore uncertainty in medical decision-making among senior internal medicine residents. J Gen Intern Med. 2022;37(12):3114–20. Nasmith L, Mcalpine L, Franco ED. Teaching by case discussion: a faculty development intervention. Med Teach. 1995;17(4):419–30. Primhak R. Assessment methods in respiratory medicine training in Europe: current status and future needs. Breathe, 2020. 16(1). Nair BR, et al. A systematic approach to workplace-based assessment for international medical graduates. Med J Aust. 2012;196(6):399–402. Mohanaruban A, Flanders L, Rees H. Case-based discussion: perceptions of feedback. Clin Teacher. 2018;15(2):126–31. Phillips AW, et al. Surgical Trainers' Experience and Perspectives on Workplace-Based Assessments. J Surg Educ. 2015;72(5):979–84. Barrett A et al. Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study. Springerplus, 2016. 5. Begum J, et al. Workplace-based assessment in the era of competency based medical education: Perception and experiences as a medical teacher. Med J Dr DY Patil Univ. 2023;16(2):255–9. Kirkpatrick D. Evaluation of training. In: Craig R, Bittel I, editors. Training and development handbook. New York, NY: McGraw-Hill; 1967. pp. 87–112. Ingham G, et al. GP supervisors - an investigation into their motivations and teaching activities. Aus Fam Physician. 2014;43(11):808–12. Prentice S, et al. Workplace-based assessments in postgraduate medical education: a hermeneutic review. Med Educ. 2020;54(11):981–92. Ten Cate O, et al. Medical competence as a multilayered construct. Med Educ. 2024;58(1):93–104. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Under Review Version 1 posted Reviews received at journal 31 Dec, 2025 Reviewers agreed at journal 24 Dec, 2025 Reviewers invited by journal 24 Dec, 2025 Editor invited by journal 28 Nov, 2025 Editor assigned by journal 19 Nov, 2025 Submission checks completed at journal 19 Nov, 2025 First submitted to journal 11 Nov, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-8083958","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":565344072,"identity":"50f9034f-4b15-4c9f-b7f4-1803d72dc431","order_by":0,"name":"Hannah Wu","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABD0lEQVRIiWNgGAWjYBACAxDB2AAk2BsYJBgKGHgYDgA5PERp4TkA1GJAkhaJBLAWBoJazNnPHnvwcwdD4oabjx/e+GBgJ8N3vIHxwds2BnmDA9i1WPbkpRv2ngFquZ1mbDnDIJlH8swBZsO5bQyGG3BoMTiQYybB28aQu+F2gpk0jwEzj8GNBDZpoAgjTi3n35hJ/gVpuXn8m/Qfg3oeg/sP2H8Dtdjj1HIjx0wabMsNHjNpBoPDQFsY2JiBIom4tbwxk5Ztk6ifeSan2LLH4DjQL4nNknPOSSTPxOmwHDPJt202xnzHj2+88aOi2p7v+OGDH96U2dj24dACBRLIHEg04VU/CkbBKBgFowA/AAAgfV9NvgK/3wAAAABJRU5ErkJggg==","orcid":"","institution":"University of Adelaide","correspondingAuthor":true,"prefix":"","firstName":"Hannah","middleName":"","lastName":"Wu","suffix":""},{"id":565344073,"identity":"42a4172d-3a01-4ed4-8344-0862635931d4","order_by":1,"name":"Karan Varshney","email":"","orcid":"","institution":"Deakin University","correspondingAuthor":false,"prefix":"","firstName":"Karan","middleName":"","lastName":"Varshney","suffix":""},{"id":565344074,"identity":"31290408-2220-4589-81ac-23b3cf7eaeb9","order_by":2,"name":"Lawrie McArthur","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Lawrie","middleName":"","lastName":"McArthur","suffix":""},{"id":565344075,"identity":"a60d3e3e-1aed-45e9-aef0-a861e029debf","order_by":3,"name":"Nigel Stocks","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Nigel","middleName":"","lastName":"Stocks","suffix":""},{"id":565344076,"identity":"e021d09c-1175-4876-9026-5437aba59ed5","order_by":4,"name":"Lucie Walters","email":"","orcid":"","institution":"University of Adelaide","correspondingAuthor":false,"prefix":"","firstName":"Lucie","middleName":"","lastName":"Walters","suffix":""}],"badges":[],"createdAt":"2025-11-11 07:53:32","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-8083958/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-8083958/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":99259493,"identity":"b2613e33-43eb-4805-8291-7a0c57fc20b6","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":67682,"visible":true,"origin":"","legend":"","description":"","filename":"scopingreviewfiguresandtables.docx","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/e9aada4c5b6c7879c9efd183.docx"},{"id":99259498,"identity":"aad38922-eb77-4ae4-86bf-190c9e6c081a","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":85378,"visible":true,"origin":"","legend":"","description":"","filename":"journalarticletextonly.docx","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/971ef41fff3fd1edb08f68d5.docx"},{"id":99319466,"identity":"e74911e6-3055-4d73-aa30-1b2e3d4a4b5f","added_by":"auto","created_at":"2025-12-31 16:37:19","extension":"json","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":7787,"visible":true,"origin":"","legend":"","description":"","filename":"aa3afd32acfd4aa196cc917469ab243a.json","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/53ac59c6d4bd1e1599a34c7b.json"},{"id":99259494,"identity":"c1b84433-9bcc-4838-9474-a607026f8e24","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"xml","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":200099,"visible":true,"origin":"","legend":"","description":"","filename":"aa3afd32acfd4aa196cc917469ab243a1enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/94f03c1944aeeec856ff0e7a.xml"},{"id":99259488,"identity":"0503d3bd-04e7-43bd-92b0-918fa9723328","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"jpeg","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1074,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/43ac3a240286055430651c6f.jpeg"},{"id":99259491,"identity":"088e6bd0-8a35-478e-b662-b482ffa11f79","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"jpeg","order_by":6,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":1074,"visible":true,"origin":"","legend":"","description":"","filename":"floatimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/20ebeebb3da9a30f64641acc.jpeg"},{"id":99320035,"identity":"13d5ea12-416c-41fd-b525-bc87457a7604","added_by":"auto","created_at":"2025-12-31 16:38:06","extension":"jpeg","order_by":7,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":71777,"visible":true,"origin":"","legend":"","description":"","filename":"groupimage1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/e42edb7a054cdd0aa0a1826c.jpeg"},{"id":99259489,"identity":"f9f2f4fb-0461-4422-8b86-7cb775f68d92","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"png","order_by":8,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":935,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/65d5c3d86323eecd62a44d51.png"},{"id":99318384,"identity":"69867519-e6b2-4618-bc69-3f287f508cee","added_by":"auto","created_at":"2025-12-31 16:32:57","extension":"png","order_by":9,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":166032,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/6dba79dbc4b2044be9682add.png"},{"id":99259497,"identity":"83f453f4-7588-404e-9b22-b74e010874bb","added_by":"auto","created_at":"2025-12-31 01:12:32","extension":"png","order_by":10,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":935,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinefloatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/1ba03c0f217df6451d98d49b.png"},{"id":99259499,"identity":"c7ae6a2f-eaf1-482c-bac4-d9133954ecbe","added_by":"auto","created_at":"2025-12-31 01:12:33","extension":"png","order_by":11,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":17746,"visible":true,"origin":"","legend":"","description":"","filename":"Onlinegroupimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/ec618df386248e1941cccc53.png"},{"id":99259501,"identity":"d3763158-b0e2-4073-bbc9-5cc40bbbcbac","added_by":"auto","created_at":"2025-12-31 01:12:33","extension":"xml","order_by":12,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":195768,"visible":true,"origin":"","legend":"","description":"","filename":"aa3afd32acfd4aa196cc917469ab243a1structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/3b83f5d72cfebae524db4f4e.xml"},{"id":99259500,"identity":"d8baa2d8-6e43-42b0-90e1-cbbd7562a4e3","added_by":"auto","created_at":"2025-12-31 01:12:33","extension":"html","order_by":13,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":217490,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/1a3abc7009fa9369c2c5454e.html"},{"id":99318753,"identity":"35d4fba1-96b7-4a70-8c1d-34b649694e74","added_by":"auto","created_at":"2025-12-31 16:34:12","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":1134730,"visible":true,"origin":"","legend":"\u003cp\u003ePreferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram.\u003c/p\u003e","description":"","filename":"floatimage2.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/c62ddd609a93b492c721fbe3.jpeg"},{"id":99323973,"identity":"f12de29c-b3ad-4ef8-98dd-064fec8fa70e","added_by":"auto","created_at":"2025-12-31 16:46:47","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2251195,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-8083958/v1/b54d512d-675d-4156-a4cf-b31c17eedb55.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Utility of Case-Based Discussions as Workplace Based Assessments in Medical Training: A Scoping Review","fulltext":[{"header":"Introduction","content":"\u003cp\u003eWork integrated learning is the foundation of medical education, at all levels of training, where learners develop their expertise working alongside more senior clinicians while making authentic contributions to patient care. Case-based discussions (CbDs) are workplace-based assessment (WBA) tools that allow assessors to explore a trainee\u0026rsquo;s thinking and clinical reasoning [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. The term \u0026lsquo;CbD\u0026rsquo; is poly-semantically ambiguous. It can include chart stimulated recall used by the American Board of Emergency Medicine as early as the 1980s [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e], and random case analysis (RCA) where the case is usually chosen by the assessor. The term is also commonly used to describe various clinical and non-assessment contexts in medical education such as group discussion about a case scenario. The lack of definition and clarity of terminology is a source of confusion for students, educators, and researchers. For the purposes of this scoping review, CbD is defined as \u0026ldquo;\u003cem\u003eA structured workplace-based assessment interaction between a learner and supervisor/educator, based on a real patient case\u003c/em\u003e.\u0026rdquo;\u003c/p\u003e\n\u003ch3\u003eAssessment Utility\u003c/h3\u003e\n\u003cp\u003eThe utility of an assessment, such as CbDs, can be considered through van der Vleuten and Schuwirth\u0026rsquo;s Assessment Utility Model which recognises competing attributes of assessment utility including: validity (including reliability), educational impact, acceptability, practicability and cost-effectiveness [\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. Several recent systematic reviews have considered the utility of WBAs including CbD in medical education.\u003c/p\u003e \u003cp\u003eA 2015 systematic review of the utility of CbD, which specifically excluded studies where CbD was used in combination with other assessment methods, concluded that there was a lack of compelling evidence for the use of patient\u0026rsquo;s charts to evaluate medical trainees [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e]. In a subsequent systematic review of the usefulness of WBAs in surgical training, only two out of 23 studies focussed on CbD [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. These reviews assume all CbD assessment tools have similar qualities and do not differentiate between learners at different levels of expertise such as (novice) early clinical medical student, senior medical student, prevocational trainee and specialist vocational trainee, and (mastery) qualified specialists. The aim of this scoping review is to address this lack of nuance and examine different CbD tools and their utility as WBAs throughout each stage of medical training.\u003c/p\u003e"},{"header":"Methods","content":" \u003cdiv id=\"Sec3\" class=\"Section2\"\u003e \u003cp\u003eThe scoping review was conducted in accordance with the JBI methodology for scoping reviews [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e], and in line with the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) [\u003cspan additionalcitationids=\"CR9\" citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. The protocol was registered with Open Science Framework. Review registration number: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.17605/OSF.IO/JD86Y\u003c/span\u003e\u003cspan address=\"10.17605/OSF.IO/JD86Y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e \u003cp\u003eArticles were included where they reported on CbDs, including chart stimulated recall (CSR), and random case analysis (RCA), used throughout medical training, from undergraduate medicine to postgraduate specialty training. Eligible participants included: medical students, doctors who are either trainees (prevocational or vocational) or qualified specialists, where these learners were assessed by doctors. Exclusions include: discussion of case scenarios used as a group educational activity or where the learner was not involved in the patient\u0026rsquo;s care. Original research articles and literature review articles published in English were included in the study. There were no geographical location or publication date restrictions.\u003c/p\u003e \u003cp\u003eSearch strategies were developed for the PubMed, Scopus (Elsevier), Embase (Ovid), Web of Science (Clarivate), Cumulated Index to Nursing and Allied Health (EBSCO), Education Resources Information Center (ProQuest), PsycInfo (Ovid), and Cochrane Central Register of Controlled Trials databases (Table\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e). The literature search was undertaken in March 2024. The reference lists of all included sources of evidence were screened for additional studies.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003esearch strategy\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"3\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eSearch\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eQuery\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eRecords retrieved\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCase-based discussion*[tw] OR Chart stimulated recall*[tw] OR Random case analysis[tw] OR Case discussion*[tw]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2,093\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eGeneral practi*[tw] OR Primary care [tw] OR\u003c/p\u003e \u003cp\u003eMedicine [tw] OR Physician*[tw] OR Trainee* [tw] OR Clinician* [tw] OR Medical student*[tw] OR\u003c/p\u003e \u003cp\u003eMedical education [tw] OR \"Medicine\"[mh] OR\u003c/p\u003e \u003cp\u003e\"Physicians\"[mh] OR \"Students, medical\"[mh] OR\u003c/p\u003e \u003cp\u003e\"Educational measurement \"[mh] OR \"Formative feedback\"[mh] OR \"Education, medical\"[mh] OR \"Clinical decision making\"[mh]\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2,965,756\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003e1\u0026thinsp;+\u0026thinsp;2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003e(Case-based discussion*[tw] OR Chart stimulated recall*[tw] OR Random case analysis[tw] OR Case discussion*[tw]) AND (General practi*[tw] OR Primary care [tw] OR\u003c/p\u003e \u003cp\u003eMedicine [tw] OR Physician*[tw] OR Trainee* [tw] OR Clinician* [tw] OR Medical student*[tw] OR\u003c/p\u003e \u003cp\u003eMedical education [tw] OR \"Medicine\"[mh] OR\u003c/p\u003e \u003cp\u003e\"Physicians\"[mh] OR \"Students, medical\"[mh] OR\u003c/p\u003e \u003cp\u003e\"Educational measurement \"[mh] OR \"Formative feedback\"[mh] OR \"Education, medical\"[mh] OR \"Clinical decision making\"[mh])\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e1,149\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFollowing the search, all identified citations were collated and uploaded into \u003cem\u003eCovidence\u003c/em\u003e software [\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e] and duplicates removed. Following a pilot test, titles and abstracts were screened by two independent reviewers to assess the inclusion criteria for the review. Potentially relevant papers were retrieved in full and independently assessed against the inclusion criteria by two reviewers.\u003c/p\u003e \u003cp\u003eAn excel spreadsheet was created to organise information extracted from articles. One line was allocated to each article with columns relating to specific attributes: the participants, context, assessment tool, study methods, and key findings related the factors in the utility framework [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. Messick\u0026rsquo;s unified validity framework was used to evaluate the validity of the assessment methods [\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e]. Data was extracted from articles by two independent reviewers and then compared. Any disagreements in article selection or data extraction were resolved through discussion or with an additional reviewer.\u003c/p\u003e \u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003eA total of 4357 articles were screened, 131 were subject to full-text review. Finally, 43 articles were determined eligible for inclusion in this scoping review as outlined in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e PRISMA-ScR flow diagram [\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e. Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review (PRISMA-ScR) flow diagram.\u003c/p\u003e \u003cp\u003e \u003cb\u003eStudy characteristics\u003c/b\u003e \u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eOf the 43 articles, 18 were set in the UK, five in United States of America, four in Australia, one across Australia and New Zealand combined, four in Canada, two in Pakistan, two in India, one each in Ireland, Europe, Turkey, and Malaysia. Sixteen articles were published between 2015 and 2019. There were 3 systematic reviews [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e], and 6 qualitative studies. Quantitative studies included: 7 prospective cohort, 7 cross-sectional and 10 retrospective cohort studies. Mixed methods studies included 1 prospective cohort and 9 cross-sectional studies. The clinical disciplines for the original research were predominantly general practice (13 studies).\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eCharacteristics of reviews\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"13\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c8\" colnum=\"8\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c9\" colnum=\"9\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c10\" colnum=\"10\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c11\" colnum=\"11\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c12\" colnum=\"12\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c13\" colnum=\"13\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colspan=\"5\" nameend=\"c7\" namest=\"c3\"\u003e \u003cp\u003eNumber of papers\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c10\" namest=\"c8\"\u003e \u003cp\u003ePapers\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c13\" namest=\"c11\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"10\" nameend=\"c10\" namest=\"c1\"\u003e \u003cp\u003eREGION AND COUNTRY OF STUDY\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c13\" namest=\"c11\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c3\" namest=\"c1\"\u003e \u003cp\u003eEurope\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c7\" namest=\"c4\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c3\" namest=\"c1\"\u003e \u003cp\u003eUnited Kingdom\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c7\" namest=\"c4\"\u003e \u003cp\u003e18\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eAryal 2020, Awad 2015, Bodgener 2011, Bodgener 2017, Brittlebank 2013, Crossley 2011, Eardley 2013, Evans 2005, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Mitchell 2011, Mohanaruban 2017, Mynors-Wallis 2011, Nicol 2018, Phillips 2015, Phillips 2016, Shaw 2014\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c3\" namest=\"c1\"\u003e \u003cp\u003eIreland\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c7\" namest=\"c4\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eBarrett 2016\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c3\" namest=\"c1\"\u003e \u003cp\u003eEurope\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c7\" namest=\"c4\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003ePrimhak 2020\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eNorth America\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eUnited States\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eHolt 2017, Mutter 2022, Rakosfky 2023, Reddy 2018, Solomon 1990\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eCanada\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eGoulet 2007, Jennett 1995, Nasmith 1995, Norman 1993\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eAustralasia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eAustralia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eIngham 2015, Ingham 2016, Morgan 2015, Nair 2012\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eAustralia and New Zealand\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eCastanelli 2019\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eMalaysia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e \u003cp\u003eTahir 2022\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eAsia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"6\" nameend=\"c13\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eIndia\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eBegum 2023, Shrivastava 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003ePakistan\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eJafri 2020, Nadeem 2017\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eTurkey\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eYildirir 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eMultiple\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAl-Wassia 2015*, Aryal 2021*, Miller 2012*\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"12\" nameend=\"c12\" namest=\"c1\"\u003e \u003cp\u003ePUBLICATION DATE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;2010\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eEvans 2005, Goulet 2007, Jennett 1995, Nasmith 1995, Norman 1993, Solomon 1990\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003e2010\u0026ndash;2014\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e12\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eBodgener 2011, Brittlebank 2013, Crossley 2011, Eardley 2013, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Miller 2010*, Mitchell 2011, Mynors-Wallis 2011, Nair 2012, Shaw 2014\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003e2015\u0026ndash;2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e16\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAl-Wassia 2015*, Awad 2015, Barrett 2016, Bodgener 2017, Castanelli 2019, Holt 2017, Ingham 2015, Ingham 2016, Mohanaruban 2017, Morgan 2015, Nadeem 2017, Nicol 2018, Phillips 2015, Phillips 2016, Reddy 2018, Shrivastava 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003e2020\u0026ndash;2024\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAryal 2020, Aryal 2021*, Begum 2023, Jafri 2020, Mutter 2022, Primhak 2020, Rakosfky 2023, Tahir 2022, Yildirir 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"12\" nameend=\"c12\" namest=\"c1\"\u003e \u003cp\u003eLEVEL OF LEARNER\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMedical student\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003ePreclinical years\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eclinical years\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eTahir 2022\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003ePrevocational\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003e\\ Mitchell 2011, Nair 2012\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eVocational training\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e31\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAryal 2020, Awad 2015, Bodgener 2011, Bodgener 2017, Brittlebank 2013, Castanelli 2019, Eardley 2013, Evans 2005, Goulet 2007, Holt 2017, Ingham 2015, Ingham 2016, Jafri 2020, Johnson 2011, Jyothirmayi 2012, Mehta 2013, Morgan 2015, Mutter 2022, Nadeem 2017, Nasmith 1995, Nicol 2018, Phillips 2016, Primhak 2020, Rakosfky 2023, Reddy 2018, Shaw 2014, Yildirir 2023,\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eConsultant level\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eJennett 1995, Mynors-Wallis 2011, Norman 1993, Phillips 2015, Solomon 1990\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMultiple levels\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e4\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAl-Wassia 2015*, Aryal 2021*, Miller 2012*, Mohanaruban 2017\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"12\" nameend=\"c12\" namest=\"c1\"\u003e \u003cp\u003eASSESSMENT TOOL STYLE\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eBehavioural anchors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c8\" namest=\"c5\"\u003e \u003cp\u003e8\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c12\" namest=\"c9\"\u003e \u003cp\u003eAwad 2015, Castanelli 2019, Crossley 2011, Evans 2005, Goulet 2007, Norman 1993, Reddy 2018, Shrivastava 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eFeedback prompts\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c8\" namest=\"c5\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c12\" namest=\"c9\"\u003e \u003cp\u003eBodgener 2017, Ingham 2015, Ingham 2016, Jennett 1995, Rakosfky 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eNormative comparisons\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c8\" namest=\"c5\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c12\" namest=\"c9\"\u003e \u003cp\u003eEardley 2013, Jafri 2020, Johnson 2011, Jyothirmayi 2012, Mitchell 2011, Mynors-Wallis 2011, Nadeem 2017, Solomon 1990, Tahir 2022,\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eNot described\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c8\" namest=\"c5\"\u003e \u003cp\u003e21\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c12\" namest=\"c9\"\u003e \u003cp\u003eAl-Wassia 2015*, Aryal 2020, Aryal 2021*, Barrett 2016, Begum 2023, Bodgener 2011, Brittlebank 2013, Holt 2017, Mehta 2013, Miller 2010*, Mohanaruban 2017, Morgan 2015, Mutter 2022, Nair 2012, Nasmith 1995, Nicol 2018, Phillips 2015, Phillips 2016, Primhak 2020, Shaw 2014, Yildirir 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"12\" nameend=\"c12\" namest=\"c1\"\u003e \u003cp\u003eDISCIPLINE OF STUDY\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"7\" nameend=\"c7\" namest=\"c1\"\u003e \u003cp\u003eSingle discipline studies\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eGeneral practice / community medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eBodgener 2011, Bodgener 2017, Evans 2005, Goulet 2007, Ingham 2015, Ingham 2016, Jennett 1995, Morgan 2015, Nasmith 1995, Nicol 2018, Norman 1993, Shaw 2014, Shrivastava 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eMedical disciplines\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eBarrett 2016, Crossley 2011, Holt 2017, Johnson 2011, Jyothirmayi 2012, Mohanaruban 2017, Mutter 2022, Primhak 2020, Reddy 2018, Yildirir 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eSurgical disciplines\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAryal 2020, Awad 2015, Eardley 2013, Phillips 2015, Phillips 2016\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003ePsychiatry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eBrittlebank 2013, Mynors-Wallis 2011, Rakosfky 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eAnaesthetics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eCastanelli 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eChemical pathology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eJafri 2020\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eEmergency medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eSolomon 1990\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eObstetrics and Gynae\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eTahir 2022\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003ePaediatrics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eMehta 2013\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cp\u003eRadiology\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eNadeem 2017\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eNot discipline specific\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c12\" namest=\"c8\"\u003e \u003cp\u003eAl-Wassia 2015*, Aryal 2021*, Begum 2023, Miller 2010*, Mitchell 2011, Nair 2012\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"1\" nameend=\"c13\" namest=\"c13\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eSTUDY DESIGN\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c9\" namest=\"c6\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c13\" namest=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eLiterature review\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e3\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eAl-Wassia 2015, Aryal 2021, Miller 2010\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eQualitative studies\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e6\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eIngham 2015, Ingham 2016, Jennett 1995, Jyothirmayi 2012, Nicol 2018, Phillips 2016\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eQuantitative\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eProspective cohort\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eBrittlebank 2013, Mynors-Wallis 2011, Nadeem 2017, Nasmith 1995, Nicol 2018, Norman 1993, Reddy 2018\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eCross-sectional\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e7\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eBegum 2023, Evans 2005, Johnson 2011, Morgan 2015, Phillips 2015, Tahir 2022, Yildirir 2023\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eRetrospective cohort\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e10\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eAwad 2015, Barrett 2016, Bodgener 2017, Castanelli 2019, Crossley 2011, Eardley 2013, Goulet 2007, Mitchell 2011, Shaw 2014, Solomon 1990,\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003e\u003cb\u003eMixed methods\u003c/b\u003e\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eProspective cohort\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eNair 2012\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c4\" namest=\"c1\"\u003e \u003cp\u003eCross-sectional\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c6\" namest=\"c5\"\u003e \u003cp\u003e9\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c11\" namest=\"c7\"\u003e \u003cp\u003eAryal 2020, Bodgener 2011, Holt 2017, Jafri 2020, Mehta 2013, Mohanaruban 2017, Mutter 2022, Primhak 2020, Shrivastava 2019\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"2\" nameend=\"c13\" namest=\"c12\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e\n\u003ch3\u003e*Review articles\u003c/h3\u003e\n\u003cdiv id=\"Sec6\" class=\"Section2\"\u003e \u003ch2\u003eAssessment tools\u003c/h2\u003e \u003cp\u003eEight original research studies used behavioural anchors,[\u003cspan additionalcitationids=\"CR17 CR18 CR19 CR20 CR21 CR22\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e] while five studies focussed on providing feedback prompts to supervisors. [\u003cspan additionalcitationids=\"CR25 CR26 CR27\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] Nine studies used tools with normative comparisons such as below, at or above expected standard for the level of trainee.[\u003cspan additionalcitationids=\"CR30 CR31 CR32 CR33 CR34 CR35 CR36\" citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e] The majority of studies did not specifically describe the assessment tool.\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eLearner, trainers and disciplines\u003c/h3\u003e\n\u003cp\u003eCbD within vocational training was described in 31 articles [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan additionalcitationids=\"CR25\" citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan additionalcitationids=\"CR29 CR30 CR31\" citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan additionalcitationids=\"CR39 CR40 CR41 CR42 CR43 CR44 CR45 CR46 CR47 CR48\" citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR49\" class=\"CitationRef\"\u003e49\u003c/span\u003e]. No articles described CbD undertaken by transition-to-clinical (novice) medical students, one studied clinical medical students [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e], two involved prevocational trainees such as foundation year or intern,[\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e] one included both pre-vocational and vocational trainees [\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e], and five involved fully qualified clinician .[\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e, \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e].\u003c/p\u003e \u003cdiv id=\"Sec8\" class=\"Section2\"\u003e \u003ch2\u003eUtility\u003c/h2\u003e \u003cp\u003eThe review found 23 original research studies reporting on validity.[\u003cspan additionalcitationids=\"CR17\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan additionalcitationids=\"CR21 CR22 CR23 CR24 CR25 CR26\" citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan additionalcitationids=\"CR34 CR35 CR36 CR37 CR38\" citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e, \u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e, \u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e] Another 17 reported on educational impact,[\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan additionalcitationids=\"CR32\" citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e, \u003cspan additionalcitationids=\"CR39 CR40 CR41 CR42\" citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e, \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e, \u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e] 18 papers described practicability,[\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan additionalcitationids=\"CR31\" citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan additionalcitationids=\"CR39\" citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e, \u003cspan additionalcitationids=\"CR51 CR52 CR53\" citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e] seven papers described acceptability.[\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan additionalcitationids=\"CR39\" citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e, \u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e] Four studies described cost but only hinted at cost-effectiveness.[\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e, \u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e, \u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e]\u003c/p\u003e \u003cp\u003e \u003cb\u003eValidity\u003c/b\u003e \u003c/p\u003e \u003cp\u003eFace validity describes the subjective assessment of whether CbD appears to measure the constructs it is intended to measure. Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e illustrates that when assessment tools used feedback prompts or normative comparisons, CbDs were seen as a useful component of assessment of competence, offering greater capacity to unpack thinking than consultation observations [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e]. Face validity was reportedly improved with recency of consultation, an appropriate setting with adequate time for assessment, and assessor training [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. In one study where the assessment tool was not described, paediatric trainees described assessments as more beneficial in discussions about more challenging cases [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. Only one study assessing face validity described an assessment tool with behaviourally anchored descriptors, however face validity of this tool was only described in terms of expert opinions of the domains and scale descriptors [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003eValidity study findings\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"5\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e\u0026nbsp;\u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDescription of CbD tool\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAuthor/ Setting\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eLearners\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003eDescription of validity\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003eSTUDIES OF FACE VALIDITY\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eBehaviourally anchored scale\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCbD tool had five domains related to a specific EPA marked against behaviourally anchored descriptors (1\u0026ndash;10).\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eShrivastava 2019\u003c/p\u003e \u003cp\u003eIndia\u003c/p\u003e \u003cp\u003eCommunity medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e3 community medicine trainees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCopies of the assessment tools were sent to seven subject experts, who universally agreed that CbD tool descriptors and scale content validity matched expected EPAs.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eFeedback prompts\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eFormative random chart assessment tool with prompts for feedback against domains including:\u003c/p\u003e \u003cp\u003ehistory taking, examination,\u003c/p\u003e \u003cp\u003eclinical reasoning and management etc.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eIngham 2016\u003c/p\u003e \u003cp\u003eAustralia\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e10 GP trainees and medical educators\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eThe two formative assessment approaches (RCA and direct observation) provide complementary insights. RCA of less value when consultations were not recent, as registrars could not recall clinical reasoning.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eIngham 2015\u003c/p\u003e \u003cp\u003eAustralia\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e10 GP trainees, and 8 medical educators\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eRCA reportedly enabled overview of complexity, scope of practice and types of cases managed. Increased medical educators\u0026rsquo; confidence in discriminating high performers. The ability to look back through the notes in RCA enabled educators to uncover how a registrar managed a patient when not observed and over a number of consultations.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSemi-structured interview addressing: problem identification,\u003c/p\u003e \u003cp\u003ehistory, physical, treatment, and follow-up.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eJennett 1995\u003c/p\u003e \u003cp\u003eCanada\u003c/p\u003e \u003cp\u003eFamily medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e20 family physicians\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eChart-stimulated recall measured clinical reasoning better than chart audit alone.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eNormative comparison\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEight domains of good psychiatry practice. Likert scale 0\u0026ndash;5 (fails to meet standards to excels).\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMynors-Wallis 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003ePsychiatry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e86 psychiatry consultants\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003e87% thought that case-based discussion could be a useful component for revalidation.\u003c/p\u003e \u003cp\u003eAverage scores over the eight domains increase with the number of attempts.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eAssessment tools not described.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\" morerows=\"1\" rowspan=\"2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMehta 2013\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003ePaediatrics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e26 paediatric trainees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTrainees felt challenging cases resulted in more beneficial feedback (through reflective practice). To ensure validity trainees reported assessor commitment, training, appropriate setting and adequate time needed.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBodgener 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e45 GP trainees, 30 GP trainers\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eAssessment was of value in demonstrating thinking and partially assessed performance. Structured assessment could focus on specific issues. Consultation observations considered a better as CbD could be gamed with adequate preparation and background reading.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003eSTUDIES OF CONSTRUCT VALIDITY\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eBehavioural anchors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eForm consists of eight items. Each item is rated as either D: development required, S: satisfactory, O: outstanding or N: not assessed.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAwad 2015\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eSurgery (ENT)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e46 surgical trainees\u003c/p\u003e \u003cp\u003e1456 Case-based discussions\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eGrades were significantly different between core training 88.7% (CI: 87.4\u0026ndash;89.9%), and specialist training 95.3% (CI: 94.7\u0026ndash;95.9%). CbDs were highly sensitive for competence (99%), yet not specific.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eAssessors received training. Charts were assessed for quality of record keeping and quality of care based on explicit criteria. Marked as satisfactory\u0026thinsp;=\u0026thinsp;1, unsatisfactory or NA\u0026thinsp;=\u0026thinsp;0\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eGoulet 2007\u003c/p\u003e \u003cp\u003eCanada\u003c/p\u003e \u003cp\u003eFamily medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e25 family physicians,\u003c/p\u003e \u003cp\u003e16 to 48 charts were assessed for each.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eConcordance between chart review alone and that of chart review with CSR interview was 75% for chart keeping, 69% for clinical\u003c/p\u003e \u003cp\u003einvestigation, 81% for diagnostic accuracy, and 74% for treatment plan.)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRating forms with seven-point scales and checklists to rate and score each domain: eg data acquisition, problem solving, management.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNorman 1993\u003c/p\u003e \u003cp\u003eCanada\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eOntario primary care physicians\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCSR discriminated among subgroups of clinicians: 26 randomly selected reference physicians, 20 self-referred and 37 referred by the licensing body.\u003c/p\u003e \u003cp\u003eCorrelation between CSR and other assessments eg, 0.48 with MCQs, 0.78 with standard patients, 0.76 with oral test, 0.38 with OSCEs.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eNormative comparison\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStructured grading sheet \u0026ndash; 4 case-discussion items each marked using a six-\u003c/p\u003e \u003cp\u003epoint Likert scale (very poor to excellent).\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eTahir 2022\u003c/p\u003e \u003cp\u003eMalaysia\u003c/p\u003e \u003cp\u003eObstetrics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e80 medical students\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eStudent self-assessment vs faculty assessment of case discussion) correlation coefficient of 0.307 (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01) with students underrating their performance.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCbD tool had seven items each marked using marked using a six-point Likert scale (below expectations to Above expectations for F1)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMitchell 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eFoundation year\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e1646 Foundation year doctors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eOdds ratios for association between mean WBA scores and training difficulties.\u003c/p\u003e \u003cp\u003eFor CbD (OR\u0026thinsp;=\u0026thinsp;0.39; p\u0026thinsp;=\u0026thinsp;0.002). A weak predictive value for diagnosing \u0026lsquo;in difficulty\u0026rsquo; status (0.64).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eSix competencies assessed eg clinical management) assessed using 6-point Likert scale for standard for stage of training (3\u0026thinsp;=\u0026thinsp;borderline)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eEardley 2013\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eSurgery\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2736 trainees completed 17904 CbD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eOnly 0.55% of scores below 4/6\u003c/p\u003e \u003cp\u003eNo relation between stage of training and summary rating given by assessor.\u003c/p\u003e \u003cp\u003eMeans \u0026ndash; 6.5 assessments per trainee, 20minutes assessment, 9minutes feedback.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDomains included data acquisition, problem solving, patient management, resource utilisation etc rated on an 8-point scale (\u0026thinsp;\u0026gt;\u0026thinsp;=\u0026thinsp;5\u0026thinsp;=\u0026thinsp;acceptable).\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSolomon\u003c/p\u003e \u003cp\u003e1990\u003c/p\u003e \u003cp\u003eUS\u003c/p\u003e \u003cp\u003eEmergency medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e8 ED consultants\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCorrelation between CSR and simulated patient encounters was 0.49 (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01)\u003c/p\u003e \u003cp\u003eCorrelation between recertification and oral certification (0.37, p\u0026thinsp;\u0026lt;\u0026thinsp;0.005) and written certification (0.45, p\u0026thinsp;\u0026lt;\u0026thinsp;0.05) scores.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"2\" rowspan=\"3\"\u003e \u003cp\u003eAssessment tools\u003c/p\u003e \u003cp\u003enot described.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBarrett 2016\u003c/p\u003e \u003cp\u003eIreland\u003c/p\u003e \u003cp\u003eMedicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e261 CbDs\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eLow number of assessments documenting borderline or below expectation competence. Expectations system did not explicitly reference curriculum outcomes or competencies.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eShaw 2014\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eGP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e116 GP trainees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCbD scores were not statistically sig different between cohorts passing and failing hurdle clinical skills assessment however higher numbers of CbD was associated with failing the final clinical skills assessment.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBrittlebank 2013 UK\u003c/p\u003e \u003cp\u003ePsychiatry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e220 psychiatry trainees\u003c/p\u003e \u003cp\u003e574 CbD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eHigh correlations occurred between CbD, peer assessment, and presentational assessments.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c5\" namest=\"c2\"\u003e \u003cp\u003eSTUDIES OF RELIABILITY\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eBehavioural anchors\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eTwo 11-item scoring forms were developed with either a Milestone Based Rating Scale (using Internal Medicine Milestones) or Standard Rating.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eReddy 2018 USA\u003c/p\u003e \u003cp\u003eMedicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003ePhysician trainees\u003c/p\u003e \u003cp\u003eFaculty scored\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eWith use of Milestone Based Rating Scale inter-rater reliability improved from 0.51 (with standard rating scale) to 0.61 (small effect).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eForm consists of eight items. Each item is rated as either D: development required, S: satisfactory, O: outstanding or N: not assessed.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eAwad 2015\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eENT surgery\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e46 surgical trainees\u003c/p\u003e \u003cp\u003e1456 CbD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCbD was found to be reliable (Cronbach\u0026rsquo;s a\u0026thinsp;=\u0026thinsp;0.848).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eStudy compared CbD conventional and construct-aligned scales\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCrossley 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003ePhysician\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e2448 trainees 4513 CbD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eUsing construct-aligned scales reduced CbD required from 8 to 3 for R\u0026thinsp;=\u0026thinsp;\u0026gt;\u0026thinsp;0.7\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eRating forms with seven-point scales and checklists to rate and score each domain\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eNorman 1993\u003c/p\u003e \u003cp\u003eCanada\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003eOntario primary care physicians\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTest reliability was high, ranging from 0.73 to 0.91, and all tests dis-\u003c/p\u003e \u003cp\u003ecriminated among subgroups of clinicians.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"3\" rowspan=\"4\"\u003e \u003cp\u003eNormative comparison\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eUndisclosed number of items and \u0026ldquo;overall clinical judgement\u0026rdquo; were rated from \u0026lsquo;well below expectations to \u0026lsquo;well above expectations\u0026rdquo;\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eJohnson 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eMedicine specialities\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e42 physician trainees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eTotal of 12 CbD required for reliability 0.7\u003c/p\u003e \u003cp\u003eNo CbD scores were \u0026lsquo;below\u0026rsquo; or \u0026lsquo;well below expectations\u0026rsquo;.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eEight domains of good psychiatry practice used as CbD assessment items. Likert scale 0\u0026ndash;5 (fails to meet standards to excels).\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eMynors-Wallis 2011\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003ePsychiatry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e86 psychiatry consultants\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eItem-total correlations for 8 CbD domains between 0.65\u0026ndash;0.86\u003c/p\u003e \u003cp\u003eR\u0026thinsp;=\u0026thinsp;0.84\u003c/p\u003e \u003cp\u003eThe issue of how cases are selected is important. Most doctors in the study supported random selection.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eDomains included data acquisition, problem solving, patient management, resource utilisation etc rated on an 8-point scale (\u0026thinsp;\u0026gt;\u0026thinsp;=\u0026thinsp;5\u0026thinsp;=\u0026thinsp;acceptable care.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eSolomon\u003c/p\u003e \u003cp\u003e1990\u003c/p\u003e \u003cp\u003eUS\u003c/p\u003e \u003cp\u003eEmergency medicine\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e8 ED consultants\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eReliability R\u0026thinsp;=\u0026thinsp;0.54 for 3 CSR cases\u003c/p\u003e \u003cp\u003eInter-rater reliability across 3 CSR cases was 0.70 (p\u0026thinsp;\u0026lt;\u0026thinsp;0.03).\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cp\u003eCompetences selected by the trainee from course list. Each competency marked as either: insufficient evidence, needs further development, competent or excellent.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBodgener 2017\u003c/p\u003e \u003cp\u003eUK\u003c/p\u003e \u003cp\u003eGeneral practice\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e162 CbD from 30 GP trainees.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eSignificant variance between assessors (hospital staff vs consultants vs GPs), with GPs providing lower scores.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cp\u003eAssessment tool not described.\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eCastanelli 2019\u003c/p\u003e \u003cp\u003eANZ\u003c/p\u003e \u003cp\u003eAnaesthetics\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e11,125 CbD from 1,411 trainees\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eCbD had higher reliability than DOPS and MSF With greater weighting for more reliable assessments, acceptable composite reliability for interim and final decisions is possible with nine (R\u0026thinsp;=\u0026thinsp;0.7) and 15 WBAs (R\u0026thinsp;=\u0026thinsp;0.8)\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cp\u003eBrittlebank 2013 UK\u003c/p\u003e \u003cp\u003ePsychiatry\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cp\u003e220 psychiatry trainees\u003c/p\u003e \u003cp\u003e574 CbD\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cp\u003eR\u0026thinsp;=\u0026thinsp;0.80 after 100 minutes of assessment time gathered over 4 episodes, each with different assessors.\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003e \u003cem\u003eCbD\u0026thinsp;=\u0026thinsp;case based discussions, CSR\u0026thinsp;=\u0026thinsp;chart stimulated recall, DOPS\u0026thinsp;=\u0026thinsp;direct observation of procedural skills, EPA\u0026thinsp;=\u0026thinsp;Entrustable professional activities, MCQs\u0026thinsp;=\u0026thinsp;multiple choice questions, MSF\u0026thinsp;=\u0026thinsp;multi-source feedback, OSCEs\u0026thinsp;=\u0026thinsp;observed structured clinical examinations, RCA\u0026thinsp;=\u0026thinsp;random case analysis, WBA\u0026thinsp;=\u0026thinsp;workplace-based assessment.\u003c/em\u003e \u003c/p\u003e \u003cp\u003eConstruct validity measures the extent to which a test covers all aspects of the construct it aims to measure and is often described in teams of correlations to other tests which seek to assess the same learning outcomes. Tools with behaviourally anchored scales showed high construct validity with good discrimination between candidate cohorts, reasonable concordance with chart review alone, standard patients (r\u0026thinsp;=\u0026thinsp;0.78), and oral tests (0.76) [\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e, \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. In another study where the assessment tool was not clearly described, high correlations occurred between case based discussions peer assessments and presentation assessments [\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. CbD with normative comparison did not fair as well, with poor correlations with student self-assessment, simulated patient encounters, oral and written certification scores [\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e]. There was weak predictive value of diagnosing learners in difficulty and only 0.55% of assessments scoring below 4/6 [\u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan citationid=\"CR33\" class=\"CitationRef\"\u003e33\u003c/span\u003e]. However, in one study which did not describe assessment tools, increased number of CbDs undertaken by GP trainees was associated with increased likelihood of failing [\u003cspan citationid=\"CR45\" class=\"CitationRef\"\u003e45\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eReliability of CbD tool is the likelihood that a candidate will receive the same grade for the same assessment. Overall, reliability was frequently reported in the order of 0.7 (after around 3\u0026ndash;9 case based discussions or 100 minutes of assessment) to 0.8 (after 12\u0026ndash;15 case based discussions) [\u003cspan additionalcitationids=\"CR17\" citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. In one study behavioural anchors improved reliability when compared to a normative comparison tool (from 0.51 to 0.61) [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. The type of assessor also appeared to influence reliability with one study showing lack of reliability with increasing seniority of hospital-based assessors for GP trainees compared to GP educators [\u003cspan citationid=\"CR24\" class=\"CitationRef\"\u003e24\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e\n\u003ch3\u003eEducational impact\u003c/h3\u003e\n\u003cp\u003eEducational impact was considered from the perspective of feedback received (Kirpatrick level 1), reported changes in knowledge or skills (level 2) and evidence of improvement on further testing (level 3).[\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e] Level 4 changes in organisational practice and benefits to patients was not found.\u003c/p\u003e \u003cp\u003eSeveral studies indicated that CbD was a preferred work-based assessment tool by examiners who reported they could gain deep insight, enable rich discussions and provide feedback on the learner\u0026rsquo;s consulting, specific aspects of patient care and how these could be improved in the future, including development of future learning objectives [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e, \u003cspan citationid=\"CR54\" class=\"CitationRef\"\u003e54\u003c/span\u003e]. Agility of the tool was important in enabling exploration of the trainee\u0026rsquo;s clinical reasoning and broadening their higher order thinking [\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eCbD can be described as having three phases: reporting, analysing and managing [\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e]. Trainer\u0026rsquo;s skills in providing feedback impacted on the educational value [\u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. Feedback could be seen as \u0026lsquo;tick a box\u0026rsquo; and not specific enough to support learning, particularly for less competent learners where concern was raised that supervisors were \u0026lsquo;afraid of failing\u0026rsquo; trainees [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]. When learners had many CbD to do they also described wanting to \u0026lsquo;just get them done\u0026rsquo; with little regard to the feedback received [\u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e]. Lack of follow-up by examiners was seen as a missed opportunity to consolidate learnings [\u003cspan citationid=\"CR53\" class=\"CitationRef\"\u003e53\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eChanges in attitudes and perceptions were described in a few studies, including increasing trainees\u0026rsquo; own awareness of influencing factors on treatment decisions,[\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e] aiding clinical decision-making and effecting a change in practice [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. Some studies purported to encourage reflective learning based on the case and performance in the CbD [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e, \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. In one study 23% of medical trainees undertook self-directed learning in response to their CbD, while 20% indicated they did not change their practice [\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]. More senior learners described that more challenging cases resulted in more beneficial feedback [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eEvidence of concrete behaviour change was demonstrated by radiology residents who, when compared to peers who did not undertake CbD, improved their reporting skills following weekly CbDs.[\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e] A positive impact on resident behaviour was reported following CbD with faculty (facilitators) self-reporting a change in residents\u0026rsquo; documentation skills and insight into hospital systems-based issues [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eOnly two studies described systems level benefit from undertaking an RCA activity [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. Morgan found that 29.8% of the Australian GP supervisors surveyed in their study, reported that RCA enabled identification of critical patient safety issues that resulted in a change in patient management, including delayed diagnosis, inappropriate test ordering, inadequate follow-up, and poor quality of clinical records [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. A proportion (15.8%) of trainers also described that undertaking the RCA with the trainee resulted in a change to the original management plan, suggesting improved patient care [\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cb\u003ePracticability\u003c/b\u003e \u003c/p\u003e \u003cp\u003ePracticability, including feasibility, can be considered from the perspective of how the assessment is implemented and managed within the clinical environment. Themes relating to practicability within the clinical setting included: (i) implementation, including training of assessors and orientation of learners and (ii) the need for protected time and space, where both learner and assessor can focus.\u003c/p\u003e \u003cp\u003eImplementation of CbD assessments in the clinical setting were not well described in the literature. Educators went to considerable effort to develop and refine a specific CbD tool in consultation with supervisors and learners [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e]. Several studies described efforts to orientate faculty [\u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e], improve supervisor feedback [\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e] and provide additional training to senior staff completing CbD assessments (from several hours to six months) [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan citationid=\"CR48\" class=\"CitationRef\"\u003e48\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eMany studies recognised that CbD can be time-consuming to complete, particularly if the assessment rubrics are long [\u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]. The average time taken to complete a CbD was 20\u0026ndash;25 minutes, with the total assessment time in one program designed for international medical graduates being 10 hours [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e35\u003c/span\u003e, \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. Potential pitfalls in the implementation of CbD primarily related to its time-consuming nature [\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]. Sufficient protected time was necessary for CbDs to be seen as effective [\u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]. Trainees with little power in the clinical setting found it difficult to initiate CbDs and complete the required number [\u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e].\u003c/p\u003e\n\u003ch3\u003eAcceptability\u003c/h3\u003e\n\u003cp\u003eAcceptability specifically relates to learner and assessor attitude toward and engagement in the CbD assessment process. Some studies used survey tools and found high measures of satisfaction [\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e, \u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e, \u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e, \u003cspan citationid=\"CR52\" class=\"CitationRef\"\u003e52\u003c/span\u003e] and agreement that CbD could be applied in their training context.[\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e] Other studies explored acceptance through qualitative interview [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e]. Despite its time-consuming nature, CbD was frequently seen as highly acceptable as a result of its high educational value, particularly when used as a formative tool [\u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e28\u003c/span\u003e, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e32\u003c/span\u003e, \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e38\u003c/span\u003e]. One study described that formative CbD alleviated trainees stress related to uncertainty [\u003cspan citationid=\"CR47\" class=\"CitationRef\"\u003e47\u003c/span\u003e]. In one study of surgical trainees, the benefit was felt to be greater for those at a junior stage of training [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. Trainee\u0026rsquo;s attitudes to CbD were influenced by their relationship with the assessor, high levels of engagement and their approach to feedback.[\u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e25\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e, \u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e] Time constraints had a negative impact on feedback [\u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e]. One suggestion to improve acceptability was to use a combination of ad hoc and pre-planned assessments [\u003cspan citationid=\"CR51\" class=\"CitationRef\"\u003e51\u003c/span\u003e]. The impact of case selection, including proximity to time of patient consultation, on the perceived benefit of CbD was highlighted in a few studies [\u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e26\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e].\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eCost\u003c/h2\u003e \u003cp\u003eThere were no articles that discussed the overall cost-effectiveness of CbD, although some studies discuss issues pertaining to cost. Costs have been reported more generally for WBA programs. In competency assessment of primary care physicians, the cost of a peer review activity was approximately \u003cspan\u003e$\u003c/span\u003e850 CAN for each peer assessment and feedback session, an annual cost of \u003cspan\u003e$\u003c/span\u003e935 000 CAN in 1993 [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. In an Australian postgraduate context, the estimated cost of delivering a whole WBA program (where 2 CbD were included in a total of 22 assessments over 6 months) was estimated to be \u003cspan\u003e$\u003c/span\u003e6000 AUD per international medical candidate in 2012 [\u003cspan citationid=\"CR50\" class=\"CitationRef\"\u003e50\u003c/span\u003e]. Like all assessment trade-off decisions, the cost must be balanced against the potential improvement in sensitivity in picking up learner problems [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e].\u003c/p\u003e \u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis scoping review unveiled a high degree of heterogeneity when examining CbD used throughout the entire pipeline of medical training. Validity of CbD as an assessment tool can be improved with carefully designed assessment tools with behavioural anchors, and appropriate setting with protected time. There was evidence for educational impact at low levels of Kirkpatrick\u0026rsquo;s hierarchy, with some evidence of behavioural change (level 3) and scant evidence of system and patient impact. Learner acceptability is improved by engaged assessor/trainers who provide meaningful actionable feedback and follow-up. Assessor engagement may be improved with training and longitudinal relationships with learners as found in general practice. There were insufficient studies in feasibility, and cost-effectiveness; although the time required to undertake CbD is an important factor. Fit-for-purpose assessment does not require perfect \u0026lsquo;utility\u0026rsquo;, and it is these trade-offs that educators need to consider with \u0026lsquo;purpose\u0026rsquo; at front of mind.\u003c/p\u003e \u003cp\u003eThe diversity of design of studies in this review may reflect the various contexts in which CbD is adopted, ranging from general practice (GP), medicine and subspecialties, surgery and subspecialties, to psychiatry. It is not surprising that a GP setting was the most common setting in which CbD was researched; this may reflect the inherent 1\u0026ndash;1 GP supervisor-trainee relationship in clinical practice. It may also be that CbD aptly caters for the diverse nature and style of consultations in primary care, compared with hospital training structures where supervisors and trainees are less closely connected in the work setting.\u003c/p\u003e \u003cp\u003eValidity of CbD faired well compared with other work-based assessments and they are often used as one of a suit of assessments in vocational and prevocational training. Inconsistency in CbD items and marking rubrics creates potential issues for supervisors or examiners who supervise trainees of diverse backgrounds and stages or training. When learning constructs are not clearly defined in the marksheet the validity of the assessment may be threatened. Reliability was overall acceptable (r\u0026thinsp;=\u0026thinsp;0.7\u0026ndash;0.8), but the number of CbDs needed to achieve this varied between studies (3\u0026ndash;15). Of note, the inter-rater reliability was impacted by the assessor\u0026rsquo;s clinical background and level of experience, assessor training and familiarity of standard expected of the trainee. Documenting evidence of validity is essential if the assessment is used for high stakes assessments such as certifying competency. If CbD is intended as assessment for learning, it is more relevant to optimise the educational impact.\u003c/p\u003e \u003cp\u003eMost evidence regarding educational impact pertained to levels 1 and 2 of Kirkpatrick\u0026rsquo;s hierarchy [\u003cspan citationid=\"CR55\" class=\"CitationRef\"\u003e55\u003c/span\u003e]. Measuring educational value as a reaction can be easily captured, but there are greater practical barriers for researchers when evaluating higher levels of impact such as changes in behaviour, organisational practice, and patient care. Repeating an assessment with the same trainee longitudinally may be feasible in a GP setting, but less feasible in hospital-based systems where the supervisory and assessment structure is more distributed.\u003c/p\u003e \u003cp\u003ePracticability, including implementation-related issues was not well researched. Trainee and trainer\u0026rsquo;s reactions to CbD highlighted that the time-consuming nature of CbD and the need for protected time. This raises potential issues of equity of access when learners require significant numbers of CbD to meet assessment requirements. It also highlights the effort required to engage and maintain engagement of clinical supervisors within the training system.\u003c/p\u003e \u003cp\u003eThe acceptability of CbDs was frequently high, particularly were feedback contributed to further development of clinical reasoning. While this is appropriate for postgraduate doctors who have developed this skill, medical students are not yet at a mastery stage. Their views on the acceptability of CbD need further exploration.\u003c/p\u003e \u003cp\u003eClinician time is a valuable resource and for a setting such as fee-for-service general practice, direct financial losses are tangible and cannot be disregarded. Context-specific considerations must be made for how the issue of cost-effectiveness can be overcome. In some countries like Australia, GP supervisors can received a payment for clinical placement supervision of learners e.g., \u003cspan\u003e$\u003c/span\u003e200 per half day to have a medical student. However, after accounting for reduced patient billings to enable students to have an active role in patient care, and then \u0026lsquo;blocked off\u0026rsquo; time to undertake CbDs, GP supervisors may still face an overall financial loss. GP supervisors reported motivation to teach teaching include: enjoyment, contribution to the profession and community, adding variety, and workforce succession planning, with less motivation stemming from financial reward [\u003cspan citationid=\"CR56\" class=\"CitationRef\"\u003e56\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eCost effectiveness for training institutions must account for cost of training clinician supervisors to optimise validity and educational value for learners, potentially tipping the evaluation of effectiveness of a costly investment more positively. While assessor training was consistently reported as a recommendation for improving the experience of CbD, articles stopped short of making specific recommendations for what this encompassed.\u003c/p\u003e \u003cp\u003eThe main limitations of this articles included in this review is the large number of studies which do not adequately describe the structure and focus of the assessment tool and the context of its use in the CbD,[\u003cspan citationid=\"CR57\" class=\"CitationRef\"\u003e57\u003c/span\u003e] making it impossible to fully understand utility findings. Further, in this study utility has been assessed through van der Vleuten and Schuwirth\u0026rsquo;s Assessment Utility Model which underplays the difficulty of making competence explicit and measurable [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR58\" class=\"CitationRef\"\u003e58\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eThe medical education context is increasingly moving towards longitudinal, community or workplace-based exposure early within medical degrees. The selection of assessment programmes should be informed by directly relevant evidence. This review has highlighted a striking scarcity of literature investigating the utility of CbDs in medical student learners, with only one study examining its use in clinical level medical students [\u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e], and no studies in transition-to-clinical practice level learners. It is reasonable to consider the effects of WBA in the postgraduate education space in translating this to the undergraduate context. Despite these limitations, this scoping review provides useful learnings for educational designers to optimise the utility of CbD taking into account the purpose and context of CbD in their medical education program.\u003c/p\u003e"},{"header":"Conclusions","content":"\u003cp\u003eThis scoping review has mapped the current literature on the utility of case-based discussions as a structured workplace-based assessment. Validity can be improved with carefully designed assessment tools with behavioural anchors, and appropriate setting with protected time for CbD. Educational impact including positive learner reactions and demonstrates some improvement in learner skills. Learner acceptability is improved by engaged assessor/trainers who provide meaningful, actionable feedback. The time required to undertake CbD is an important factor in practicability and acceptability. There was a striking scarcity of literature investigating the utility of CbD in medical student learners, which is discordant with the increasing emphasis on workplace-based learning and assessment in medical schools.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eEthics, and Consent to Participate: not applicable.\u003c/p\u003e \u003cp\u003e \u003cstrong\u003eConsent to publish:\u003c/strong\u003e \u003cp\u003eThe authors each consent to publish.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eFunding\u003c/h2\u003e \u003cp\u003eThis research project is supported by The Royal Australian College of General Practitioners with funding from the Australian Government under the Australian General Practice Training Program.\u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eHW wrote the scoping review protocol and registration, designed the search strategy, screened the articles, extracted data, undertook data analysis, and contributed to the manuscript.KV contributed to the scoping review protocol and registration, screened the articles, extracted data, and contributed to the manuscript.LW undertook data analysis and contributed to the manuscript.NS and LM critically reviewed the manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eThis research project is supported by The Royal Australian College of General Practitioners with funding from the Australian Government under the Australian General Practice Training Program.Thank you to Jasmine Castellano, Faculty of Health and Medical Sciences Liaison Librarian, for assistance with developing the search strategy.\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eAll data supporting the findings of this study are available within the paper and its Supplementary Information.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eSchipper S, Ross S. Structured teaching and assessment: a new chart-stimulated recall worksheet for family medicine residents. Can Fam Physician. 2010;56(9):958\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMaatsch J, et al. Predictive Validity of Medical Specialist Examinations. Final Report for Grant HS 02038-04, National Centre of Health Services Research. East Lansing, MI: Michigan State University; 1983.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNorcini J, Burch V. Workplace-based assessment as an educational tool: AMEE Guide 31. Medical teacher, 2007. 29(9\u0026ndash;10): pp. 855\u0026ndash;71.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003evan der Vleuten CP, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309\u0026ndash;17.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAl-Wassia H, et al. Using patients' charts to assess medical trainees in the workplace: a systematic review. Med Teach. 2015;37(Suppl 1):S82\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAryal K, Hamed M, Currow C. The usefulness of work-based assessments in higher surgical training: a systematic review. Int J Surg. 2021;94:106127.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAromataris E, Porritt LC, Pilla K, Jordan B Z, editors. JBI Manual for Evidence Synthesis. Editor: JBI; 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePeters MD, et al. Guidance for conducting systematic scoping reviews. Int J Evid Based Healthc. 2015;13(3):141\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTricco AC, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169(7):467\u0026ndash;73.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePeters MDJ, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synthesis. 2020;18(10):2119\u0026ndash;26.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eInnovation VH. Covidence systematic review software. Australia: Melbourne; 2024.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eVan Der Vleuten CP. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1996;1(1):41\u0026ndash;67.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMessick S. Validity.. In: Linn RL, editor Educational measurement., 1989. New York: Macmillan. pp 13\u0026ndash;103.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePage MJ, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Revista Esp Cardiologia (English Edition). 2021;74(9):790\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMiller A, Archer J. Impact of workplace based assessment on doctors\u0026rsquo; education and performance: a systematic review. BMJ, 2010. 341.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAwad Z, et al. Utilisation and outcomes of Case-based discussion in otolaryngology training. Clin Otolaryngol. 2015;40(2):86\u0026ndash;92.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCastanelli DJ, et al. The reliability of a portfolio of workplace-based assessments in anesthesia training. Can J Anesth - J Canadien D Anesthesie. 2019;66(2):193\u0026ndash;200.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCrossley J, et al. Good questions, good answers: construct alignment improves the performance of workplace-based assessment scales. Med Educ. 2011;45(6):560\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEvans A, Ormston B, Dunbar A. Development of a profile to aid feedback on trainers' skills in one-to-one teaching using random case analysis. Educ Prim Care. 2005;16(3):283\u0026ndash;94.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoulet F, et al. Assessment of family physicians' performance using patient charts - Interrater reliability and concordance with chart-stimulated recall interview. Evsluation Health Professions. 2007;30(4):376\u0026ndash;92.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNorman GR, et al. Competency Assessment of Primary Care Physicians as Part of a Peer Review Program. JAMA: J Am Med Association. 1993;270(9):1046\u0026ndash;51.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eReddy ST, et al. Preliminary Validity Evidence for a Milestones-Based Rating Scale for Chart-Stimulated Recall. J Graduate Med Educ. 2018;10(3):269\u0026ndash;75.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShrivastava SR, et al. Development, validation and use of appropriate assessment tools for certification of entrustable professional activities in community medicine to produce a competent postgraduate: A pilot study. Indian J Public Health. 2019;63(4):277\u0026ndash;81.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBodgener S, Denney M, Howard J. Consistency and reliability of judgements by assessors of case based discussions in general practice specialty training programmes in the United Kingdom. Educ Prim Care. 2017;28(1):45\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIngham G et al. ARCADO - Adding random case analysis to direct observation in workplace-based formative assessment of general practice registrars. BMC Med Educ, 2015. 15.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIngham G, Fry J, Ward B. Adding random case analysis to direct observation (ARCADO) - Updating the external clinical teaching visit to improve general practice registrar assessments. Aus Fam Physician. 2016;45(12):918\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJennett PA, et al. Patient charts and physician office management decisions: chart audit and chart stimulated recall. J Continuing Educ Health Professions. 1995;15(1):31\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRakofsky JJ, et al. A Chart-Stimulated Recall Activity to Assess Psychiatry Residents' Treatment-Based, Clinical Reasoning Skills. Acad Psychiatry. 2023;47(6):663\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEardley I, et al. Workplace-based assessment in surgical training: experiences from the Intercollegiate Surgical Curriculum Programme. ANZ J Surg. 2013;83(6):448\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJafri L et al. Fostering teaching-learning through workplace based assessment in postgraduate chemical pathology residency program using virtual learning environment. BMC Med Educ, 2020. 20(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJohnson G, et al. Assessing trainees in the workplace: results of a pilot study. Clin Med. 2011;11(1):48\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJyothirmayi R. Case-based discussion: assessment tool or teaching aid? Clinical Oncology (Royal College Radiology), 2012. 24(10): pp. 649\u0026ndash;53.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMitchell C, et al. Workplace-based assessments of junior doctors: do scores predict training difficulties? Med Educ. 2011;45:1190\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMynors-Wallis L, et al. Case-based discussion: a useful tool for revalidation. Psychiatry Bull. 2011;35(6):230\u0026ndash;4.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNadeem N, et al. Chart-stimullated Recall as a Learniing Tool for Improving Radiology Residents' Reports. Acad Radiol. 2017;24(8):1023\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSolomon DJ. and An Assessment of an Oral Examination Format for Evaluating Clinical Competence in Emergency Medicine. Acad Med, 1990. 65(9).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTahir ABIM, et al. Validity of Medical Students Self-Assessment of Proficiency in Clinical Long Case Examination. Malaysian J Med Health Sci. 2022;18(5):41\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBodgener S, Tavabie A. Is there a value to case-based discussion? Educ Prim Care. 2011;22(4):223\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBrittlebank A, et al. Workplace-Based Assessments in Psychiatry: Evaluation of a Whole Assessment System. Acad Psychiatry. 2013;37(5):301\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHolt SR, Sofair AN. Resident and Faculty Perceptions of Chart-Stimulated Recall. South Med J. 2017;110(2):142\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMehta F, Brown J, Shaw NJ. Do trainees value feedback in case-based discussion assessments? Med Teach. 2013;35(5):e1166\u0026ndash;72.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMorgan S, et al. Clinical supervision using random case analysis in general practice training. Educ Prim Care. 2015;26(1):40\u0026ndash;6.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNicol J, Cattanach K, Mhaoileoin AN. The use of case-based discussions to develop peer assessment skills among general practice trainees. Educ Prim Care. 2018;29(1):43\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePhillips A, et al. Case-based discussions: UK surgical trainee perceptions. Clin Teacher. 2016;13(3):207\u0026ndash;12.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eShaw B, et al. An investigation of factors affecting the outcome of the clinical skills assessment (CSA) in general practice specialty training. Educ Prim Care. 2014;25(2):91\u0026ndash;5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYildirir A, Turan S. Assessment and Evaluation of Cardiology Residency Training in T\u0026uuml;rkiye: A National Survey. Anatol J Cardiol. 2023;27(10):580\u0026ndash;91.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMutter M, et al. Use of chart-stimulated recall to explore uncertainty in medical decision-making among senior internal medicine residents. J Gen Intern Med. 2022;37(12):3114\u0026ndash;20.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNasmith L, Mcalpine L, Franco ED. Teaching by case discussion: a faculty development intervention. Med Teach. 1995;17(4):419\u0026ndash;30.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrimhak R. Assessment methods in respiratory medicine training in Europe: current status and future needs. Breathe, 2020. 16(1).\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNair BR, et al. A systematic approach to workplace-based assessment for international medical graduates. Med J Aust. 2012;196(6):399\u0026ndash;402.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMohanaruban A, Flanders L, Rees H. Case-based discussion: perceptions of feedback. Clin Teacher. 2018;15(2):126\u0026ndash;31.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePhillips AW, et al. Surgical Trainers' Experience and Perspectives on Workplace-Based Assessments. J Surg Educ. 2015;72(5):979\u0026ndash;84.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBarrett A et al. Profiling postgraduate workplace-based assessment implementation in Ireland: a retrospective cohort study. Springerplus, 2016. 5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBegum J, et al. Workplace-based assessment in the era of competency based medical education: Perception and experiences as a medical teacher. Med J Dr DY Patil Univ. 2023;16(2):255\u0026ndash;9.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKirkpatrick D. Evaluation of training. In: Craig R, Bittel I, editors. Training and development handbook. New York, NY: McGraw-Hill; 1967. pp. 87\u0026ndash;112.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIngham G, et al. GP supervisors - an investigation into their motivations and teaching activities. Aus Fam Physician. 2014;43(11):808\u0026ndash;12.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrentice S, et al. Workplace-based assessments in postgraduate medical education: a hermeneutic review. Med Educ. 2020;54(11):981\u0026ndash;92.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTen Cate O, et al. Medical competence as a multilayered construct. Med Educ. 2024;58(1):93\u0026ndash;104.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Case-based discussion, chart stimulated recall, medical training, medicine, workplace-based assessment","lastPublishedDoi":"10.21203/rs.3.rs-8083958/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-8083958/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground\u003c/p\u003e\n\u003cp\u003eCase-based discussions (CbDs) are a commonly used workplace-based assessment (WBA) method in postgraduate medical education. It is unclear whether they are appropriate for use with medical students, particularly early in their clinical placement experience. The objective of this scoping review is to understand the extent and type of evidence in relation to the utility of CbDs as structured WBA throughout medical training.\u003c/p\u003e\n\u003cp\u003eMethods\u003c/p\u003e\n\u003cp\u003eJoanna Briggs Scoping Review Methodology was used to examine original research articles pertaining to: (population) medical students, doctors, and supervisors; (concept) formal discussion between a single learner and a supervisor about a real patient and (outcome) educational impact, feasibility, acceptability, cost-effectiveness, reliability, and validity. Outcome measures were informed by van der Vleuten and Schuwirth’s Assessment Utility Model.\u003c/p\u003e\n\u003cp\u003eResults\u003c/p\u003e\n\u003cp\u003eThere were 4375 papers found of which 131 assessed by full-text analysis. Of the 41 original research studies and 3 systematic reviews included in the analysis. Only one included medical students. Validity of CbD was variable and highest in some studies using assessment tools with behavioural anchors. Protected time and engaged assessors improved validity. There was moderate evidence of change in attitudes and perceptions of learners. Some evidence demonstrated improved clinical reasoning and suggested improvements in patient safety. Learner acceptability was improved by engaged assessor/trainers who provided meaningful, actionable feedback and follow-up. Assessor engagement was reportedly improved with training and in studies involving longitudinal relationships with learners as found in general practice. There were insufficient studies in feasibility and cost-effectiveness, although the time required to undertake CbD was recognised as an important factor affecting both.\u003c/p\u003e\n\u003cp\u003eConclusion\u003c/p\u003e\n\u003cp\u003eWhilst there was evidence of positive educational impact for CbD, there was a striking scarcity of literature investigating CbDs in medical student learners, which is discordant with the increasing emphasis on workplace-based learning in the contemporary medical education context.\u003c/p\u003e","manuscriptTitle":"Utility of Case-Based Discussions as Workplace Based Assessments in Medical Training: A Scoping Review","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-12-31 01:12:28","doi":"10.21203/rs.3.rs-8083958/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2025-12-31T23:15:41+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"1485583741303472536913842950208268005","date":"2025-12-24T23:35:02+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-12-24T12:55:32+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-11-28T13:14:37+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-11-19T07:48:30+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-11-19T07:45:47+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2025-11-11T07:49:25+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"77d99607-87f4-4e08-80e5-732d9817ce54","owner":[],"postedDate":"December 31st, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-12-31T01:12:28+00:00","versionOfRecord":[],"versionCreatedAt":"2025-12-31 01:12:28","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-8083958","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-8083958","identity":"rs-8083958","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}