Design and Validation of an integrated Objective Structured Clinical Examination (i-OSCE) for non-surgical aesthetics program

Design and Validation of an integrated Objective Structured Clinical Examination (i-OSCE) for non-surgical aesthetics program | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Design and Validation of an integrated Objective Structured Clinical Examination (i-OSCE) for non-surgical aesthetics program Narendra Kumar, Eqram Rahman This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-685351/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Introduction : The Objective Structured Clinical Examination (OSCE) is a popular and practical method for evaluating trainee physicians’ competencies. To help assess non-surgical aesthetics students’ critical thinking and relevant abilities, we developed a novel assessment tool based on the OSCE, the i-OSCE (Integrated objective structured Clinical examination). Methods : Initially, an expert panel consisting of 5 Aesthetic Practitioners with over fifteen years of experience and a senior clinical academic were selected to develop a blueprint for i-OSCE. Through this blueprint, essential qualities and skills were identified for the assessment. To ensure the process standardisation, training workshops for examiners and simulated patients were organised. The final i-OSCE consisted of 12 stations (four clinical, four critical thinking, and four rest stations lasting 180 minutes. Results: The Interclass correlation coefficient between the station checklist items was 0.946 (average measure upper bound 0.916, lower bound 0.968; p < 0.00), considered to be significant. The Inter-Item Correlation Matrix among the clinical station checklist and critical thinking checklist items also showed statistical significance. The Pearson correlation coefficient (PCC) used to ascertain the correlation between checklist rating and global rating, yielding a high correlation (0.80 to 0.934). Conclusion : The i-OSCE has been proven to be a useful and reliable assessment tool to evaluate clinical competence and critical thinking in non-surgical aesthetics education. Educational Philosophy and Theory OSCE Non-Surgical Aesthetics Clinical Examination Critical Thinking Clinical Reasoning Introduction Assessment and evaluation are critical steps in medical education and rely on selecting a proper and robust instrument. Appropriate assessment tool helps to determine the effectiveness of educational programmes and ensures that the future clinicians are competent and suitable for independent clinical practice. However, the currently used assessment tools are insufficient to test the learners’ knowledge, skills, behaviour, and critical thinking abilities holistically. In such cases, using the ‘test battery’ approach becomes more practical to use a mix of assessment tools for measuring an array of learning domains ( 1 , 2 ). Traditionally, clinical assessment strategies comprise a combination of ‘short’ and ’long case’ evaluations. However, criticism about its low reliability ( 3 ) and modern-day constraints such as increased litigation and student appeals ( 4 ) have led institutions to focus on exams that produce trustworthy, more easily defendable outcomes. Accordingly, conventional assessment strategy evolved to overcome the challenges of traditional methodologies, such as reliance on the patient’s performance, the examiner’s bias, the non-standardised grading scheme, and the candidate’s actual performance; the assessment strategy went through an evolution process. Consequently, the assessment process became standard, and the number of variable affective students’ performance was reduced and paved the path favouring the introduction of objective structured clinical examination (OSCE), the “gold standard” for clinical assessments globally. It aims to examine the skills and ability to understand that the assessment results will reflect the trainee physicians’ day-to-day clinical performance in real-life scenarios. In various research studies, OSCEs have been helpful in terms of reliability and validity( 5 ). However, the long examination time is a cause of concern for the trainee physicians and costs to the program directors ( 6 ). OSCE mainly focuses on assessing affective, cognitive, and psychomotor learning domains. Nonetheless, performance is affected by various other factors such as knowledge to apply in real-life scenarios; non-clinical skills (decision-making, teamwork, resource management, planning, and critical thinking); attitudes; environment; emotional state; physical state; and personality traits. The drawback of OSCE is that it cannot be easily used to measure non-clinical skills ( 2 ) In clinical education, critical thinking skills are measured by high-fidelity patient simulations, “California critical thinking skills test, California critical thinking disposition inventory, Del Bueno’s performance-based development system, health science reasoning test”, and Watson-Glaser critical thinking appraisal. However, these are limited in their lack of measuring the particular aspect of health profession-related attributes, inability to evaluate medical professionals’ practical reality, and effectively assess psychometric properties ( 7 , 8 ). Critical appraisal is a subcategory of critical thinking, which specifies the ability to make clinical decisions by research evidence. Various studies have concluded that critical thinking can be refined, and without this essential ability, there can be drastically negative ramifications on trainee physicians’ decisions. There has been evidence of a direct correlation between critical thinking and academic success; unfortunately, many trainee physicians struggle on tests explicitly measuring it ( 9 ). While evaluating critical thinking strategies, there is a prominent spotlight on evidence-based practice and its role in education. Numerous systematic reviews have inferred that clinically integrated assessment methods are needed to improve further evidence-based practice skills ( 10 ). Critical appraisal has been incorporated into some of the high-stake professional and fellowship examinations where physicians are tested to assess their ability to judge a clinical paper in a short time based on its research design, result and whether to consider this to change one’s clinical practice. However, there is no evidence in the literature reporting the development and implementation of an evaluation tool for assessing clinical skills, analytical thinking, and non-clinical skills in the NSA educational program. Therefore, the current study aims to develop and validate an integrated objective structured clinical examination (i-OSCE) by integrating clinical and critical thinking stations for the NSA postgraduate program. Methods OSCE Station Blueprinting Blueprinting is the standard process of mapping the intended learning outcomes, which comprise knowledge and understanding, intellectual, practical, affective, and psychomotor skills relating to the postgraduate curriculum on NSA, with the knowledge and skill competencies to be tested in individual stations. An ‘expert panel’ was formed consisting of five aesthetic practitioners and a senior clinical academic with over fifteen years of experience. A consensual and ceaseless approach was adopted to identify the tasks to be assessed, which are essential and relevant to the NSA practice, thereby validating the content of i-OSCE (Table 1). Table 1: OSCE Station Blueprinting Content development and Validation A 2-day OSCE writing workshop was conducted for 15 aesthetic practitioners and clinical academicians in the presence of 3 expert facilitators, divided into three small groups. After a brief and structured presentation on OSCE, all the groups had a facilitator lead practice session to construct each station’s case scenarios. It was followed by critical feedback from the facilitator to the participants. The three working groups met regularly to construct case scenarios, candidate instruction, standardised patient information sheet, and, most importantly, the marking sheet where the entire scenario was deconstructed to make a performance checklist effectively to match the blueprint theme. Finally, the expert panel was reconvened again to review the constructed cases with the checklists and parity of competencies across the cases. Fifty stations (25 Clinical and 25 Critical thinking) were selected to store in the repository managed by an OSCE administrator. Clinical Stations These stations consisted of a simulated scenario of consultation in facial aesthetics. Here, candidates must take an appropriate history, clinical photography, and facial assessment to reach a specific, accurate diagnosis. Either standardised patients or patient actors were utilised within these stations. Any clinical examination skills relevant to facial aesthetics were subsidised to fit the station’s time limit. Candidates were given a brief history and asked to perform (either all or some aspects) a clinical examination and discuss it with the patient. At the end of the station, candidates were subsequently asked to summarise their findings or provide a brief management plan, including its justification, to the examiner. The other stations developed would assess candidates’ professionalism and communication skills. Critical Thinking (CT) Stations Twenty-five CT stations were created, where candidates were asked to critically appraise its validity and reliability (formulating PICO, review methodology and critical analysis of the discussion), whether the article published in a peer-reviewed journal and decide to adopt this into clinical practice (applicability). Marking scheme Every separate checklist score was weighed based on the allotted task’s significance as the station author deemed, which later reached further agreement from an expert panel in a station review meeting. Finally, each station received an independent standardisation to create the pass marks with the help of the borderline regression method, which utilised a combination of the checklist score and the examiner’s single 3 points global rating (clear pass, borderline, or clear fail) (Table 2). Table 2: Example of a checklist/marking scheme Assessor Training All the examiners participated in an hour-long orientation program to familiarise themselves with the OSCE setting, competency testing, and scoring guidelines. Further, they were provided with a guide describing the definitions of the competencies, checklist, global ratings. Standardised Patient (SP) Training Healthy volunteers were recruited with the help of a modelling agency to act as ‘simulated’ patients for all the stations. They went through coaching conducted by professional medical actors and clinical and communication skills experts. SPs were given a task, particularly to the station, and they practised until they played their roles consistently. As they were also responsible for completing their part of the checklist, a calibration video was shown to practice marking and debriefing. Pilot Study A pilot study was conducted with ten aesthetic practitioners to examine the feasibility of i-OSCE. Planned eight stations were run with one examiner and one observer. Finally, the results were reviewed, including the feedback from individual stations to amend i-OSCE documents for final implementation during the summative examination. Final i-OSCE The final examination was conducted comprised four clinical and four critical thinking stations of 15 minutes each and four rest stations with forty trainee physicians. The total run for the exam was 180 minutes. Statistical Analysis For an assessment tool to be accepted as reliable and valid, the most widely used statistical measurement is Cronbach’s alpha (11). However, some studies argue that it should not be used for internal reliability as sole measurement, as it is directly proportionate to the examination length; therefore, it indicates the station’s stability, not the internal consistency (12,13). However, concurrent use of Pearson correlation coefficient (PCC) or Spearman’s rank correlation helps overcome the issue (14). Therefore, PCC was used to investigate the strength of correlation between utilising the checklist and the global rating (clear pass, borderline, or clear fail), which helped provide a measure of the validity of the marking criteria used. For calculating the interrater reliability (IRR), Cronbach’s alpha was used through two-way mixed effects; intra-class correlations (ICC) for consistency and internal reliability. For interpretation of ICCs, Cicchetti’s classification (IRR less than 0.40 is poor; 0.40–0.59 is fair; 0.60–0.74 is good; 0.75–1.00 is excellent) was used (15). Moreover, content validity was measured with the help of experts. Statistical analysis was carried out by using IBM SPSS Statistics for Mac, Version 27.0 (IBM Corp. Armonk, NY, USA). Results The Interclass correlation coefficient between the station checklist items was 0.946 (average measure upper bound 0.916, lower bound 0.968; p<0.00), considered to be significant (Table 3). The Inter-Item Correlation Matrix among the clinical station checklist items and critical thinking checklists also showed statistical significance (Table 4). The Pearson correlation coefficient (PCC) used to ascertain the correlation between checklist rating and global rating (Table 5), yielding a high correlation (0.80 to 0.934). Table 3: Intraclass Correlation Coefficient Table 4: Inter-Item Correlation Matrix of station checklist Table 5: Checklist Vs Global Assessment Correlation Station Pearson Correlation 1 0.865 2 0.82 3 0.934 4 0.8 5 0.856 6 0.848 7 0.891 8 0.875 Discussion This is the first integrated OSCE validation study combining clinical and critical thinking skills for a postgraduate NSA education to the best of the author’s knowledge. OSCE is a flexible assessment method use to evaluate competence by direct observation based on objective assessment criteria. It is composed of several “stations” where examinees are required to conduct a range of clinical tasks against required clinical competence displaying the skills and attitudes over a given duration. The OSCE has been used to assess the skills most important to the healthcare professionals’ success, such as data acquisition, interpretation, troubleshooting, engagement, and management of erratic patient behaviour, otherwise difficult to obtain during the classic clinical review ( 16 ). Miller’s framework for clinical competency development recommended four stages; “knows the facts”; “knows how to elaborate and integrate the understanding”; “shows how” they apply knowledge, skills and attitude for the patient outcome; and finally “does” employ all the skills in their independent practice to serve the community, proven to work reasonably well in medical education settings ( 17 , 18 ). Evidence suggests that the OSCE helps assess the third stage “shows how” by concentrating on the clinical skills in a safe learning environment. Critical thinking is considered to be a crucial cognitive method for the creation and utilisation of knowledge. It plays a functional role during problem-solving and decision-making in a social, clinical, or ethical context. Moreover, it is equally valuable for analysing complex data, assessing situations, and implementing the most suitable actions. In a recent article, “critical thinking is described as a cognitive process, purposeful, self-regulatory judgment that has two components of cognitive skills (interpretation, analysis, inference, evaluation, explanation, and self-regulation) and a motivational component (the disposition toward critical thinking)” ( 19 ). More focus has been put in recent years on improving higher-level thought (critical thinking and clinical reasoning) skills to help physicians retain clinical integrity and medical professionalism. More than two-thirds of the reported mistakes in diagnosis are linked to physician’s lack of critical thinking ability in the present context. Given the belief that healthcare professionals must be logical thinkers, there is no consensus on the most successful model to teach and evaluate critical thinking and clinical reasoning skills ( 8 ). Recent research, which evaluated a wide range of quantitative and qualitative competencies, including behavioural and communication skills, showed that the OSCE was valid and reliable and essential for positive educational effects. Several authors have advocated that emphasising an aim of OSCEs is to develop affability in critical thinking as a precursor to practising ( 20 ). There is no valid assessment instrument combining clinical and critical appraisal skills to evaluate safe practice in non-surgical aesthetics. Therefore, using different stations to evaluate clinical skills and critical thinking ability is beneficial in this evaluation. The various clinical skills measured were consultation skills pertaining to the NSA, knowledge of the signs of ageing and the underlying anatomy, assessment of skin quality, full-face assessment to identify treatment needs for optimal results, clinical photography, development of an efficient and optimal treatment plan, safer injection techniques, post-treatment advice, complication management, and situational judgments. During the evaluation, the critical appraisal skills measured included understanding the research relating to facial assessment, botulinum toxin science, rheology of the soft-tissue fillers and complication management. It has shown that the generalisation coefficients appear to differ significantly from 0.40 to 0.85, while the majority of these coefficients vary from 0.5–0.6 ( 6 ). In the present study, the average intraclass correlation coefficient measures range between 0.916 to 0.968, which is more than the reliability coefficient threshold of 0.8 or over. The variability in the generalizability coefficients may be attributed to the examinees’ variable performance on different OSCE stations (content specificity). I-OSCE is shown to be robust and able to test applicants for their competence to carry out multiple component tasks. Conclusion Integrated OSCE has demonstrated to be a reliable and accurate assessment tool for examining the trainee aesthetic physicians’ professional competence. This tool has objectively evaluated trainee physicians critical thinking and clinical skills, including clinical reasoning. The program directors should consider the deployment of i-OSCE along with OSPE as an assessment tool in the postgraduate curriculum for non-surgical aesthetics. Abbreviations NSA non-surgical aesthetics OSCE Objective structured clinical examination I-OSCE Integrated Objective structured clinical examination OSPE Objective structured practical examination SP Standardised patient Declarations Ethics Approval and Consent to Participate The study was approved by the Faculty Research Ethics Panel of the Faculty of Medical Science, Anglia Ruskin University, Chelmsford, UK. Consent for publication: Not applicable Availability of data and materials: The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request. Competing interest: The authors declare no conflict of interest. Funding: No funding. Author’s contribution: NK drafted and prepared the manuscript. ER supervised and edited the manuscript. NK and ER analysed and interpreted the data and approved the final manuscript. Acknowledgements: The authors would like to thank all the experts who participated in during the blueprinting process and subsequently acted as examiners. Authors would also like to thank actors who performed the role of standard patients; without their support this study would have not been possible. References 1. Tabish SA. Assessment methods in medical education. Int J Health Sci (Qassim) [Internet]. 2008 Jul;2(2):3–7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21475483 2. Khan KZ, Gaunt K, Ramachandran S, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: Organisation & Administration. Med Teach [Internet]. 2013 Sep 22;35(9):e1447–63. Available from: http://www.tandfonline.com/doi/full/10.3109/0142159X.2013.818635 3. Tey C, Chiavaroli N, Ryan A. Perceived educational impact of the medical student long case: a qualitative study. BMC Med Educ [Internet]. 2020 Dec 7;20(1):257. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-020-02182-6 4. Pell G, Roberts TE. Setting standards for student assessment. Int J Res Method Educ [Internet]. 2006 Apr;29(1):91–103. Available from: http://www.tandfonline.com/doi/abs/10.1080/01406720500537486 5. Kumar N, Bhardwaj S, Rahman E. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom : a cohort study. Adv Med Educ Pract. 2018;9:239–45. 6. Marwaha S. Objective Structured Clinical Examinations (OSCEs), psychiatry and the Clinical assessment of Skills and Competencies (CASC)Same Evidence, Different Judgement. BMC Psychiatry [Internet]. 2011 Dec 16;11(1):85. Available from: http://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-11-85 7. Nguyen K, Ben Khallouq B, Schuster A, Beevers C, Dil N, Kay D, et al. Developing a tool for observing group critical thinking skills in first-year medical students: a pilot study using physiology-based, high-fidelity patient simulations. Adv Physiol Educ [Internet]. 2017 Dec 1;41(4):604–11. Available from: https://www.physiology.org/doi/10.1152/advan.00126.2017 8. Majumder MAA, Sa B, Alateeq FA, Rahman S. Teaching and Assessing Critical Thinking and Clinical Reasoning Skills in Medical Education. In 2019. p. 213–33. Available from: http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-5225-7829-1.ch012 9. West DC, Pomeroy JR, Park JK, Gerstenberger EA, Sandoval J. Critical Thinking in Graduate Medical Education. JAMA [Internet]. 2000 Sep 6;284(9):1105. Available from: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.284.9.1105 10. Maudsley G, Strivens J. Promoting professional knowledge, experiential learning and critical thinking for medical students. Med Educ. 2000 Jul;34(7):535–44. 11. Swanson DB, Stillman PL. Use of Standardized Patients for Teaching And Assessing Clinical Skills. Eval Health Prof [Internet]. 1990 Mar 29;13(1):79–103. Available from: http://journals.sagepub.com/doi/10.1177/016327879001300105 12. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika [Internet]. 1951 Sep;16(3):297–334. Available from: http://link.springer.com/10.1007/BF02310555 13. Streiner DL. Starting at the Beginning: An Introduction to Coefficient Alpha and Internal Consistency. J Pers Assess [Internet]. 2003 Feb;80(1):99–103. Available from: http://www.tandfonline.com/doi/abs/10.1207/S15327752JPA8001_18 14. Graham JM. Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability. Educ Psychol Meas [Internet]. 2006 Dec 2;66(6):930–44. Available from: http://journals.sagepub.com/doi/10.1177/0013164406288165 15. Al-Osail AM, Al-Sheikh MH, Al-Osail EM, Al-Ghamdi MA, Al-Hawas AM, Al-Bahussain AS, et al. Is Cronbach’s alpha sufficient for assessing the reliability of the OSCE for an internal medicine course? BMC Res Notes [Internet]. 2015 Dec 19;8(1):582. Available from: http://www.biomedcentral.com/1756-0500/8/582 16. Zayyan M. Objective Structured Clinical Examination: The Assessment of Choice. Oman Med J [Internet]. 2011 Jul 25;26(4):219–22. Available from: http://www.omjournal.org/fultext_PDF.aspx?DetailsID=114&pdf=images/114_M_Deatials_Pdf_.pdf&type=pdf 17. Miller GE. The assessment of clinical skills/competence/performance. Acad Med [Internet]. 1990 Sep;65(9):S63-7. Available from: http://journals.lww.com/00001888-199009000-00045 18. Sousa AC, Wagner DP, Henry RC, Mavis BE. Better data for teachers, better data for learners, better patient care: college-wide assessment at Michigan State University’s College of Human Medicine. Med Educ Online [Internet]. 2011 Jan 14;16(1):5926. Available from: https://www.tandfonline.com/doi/full/10.3402/meo.v16i0.5926 19. Nematbakhsh M, Sharif S-M, Athari Z-S, Nasr A. Assessing critical thinking in medical sciences students in two sequential semesters: Does it improve? J Educ Health Promot. 2013;2(1):5. 20. Krusen NE, Rollins D. Design of an OSCE to Assess Clinical Competence of Occupational Therapy Students. J Occup Ther Educ [Internet]. 2019 Jan 1;3(1). Available from: https://encompass.eku.edu/jote/vol3/iss1/11 Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-685351","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":45573672,"identity":"c1082c40-8de3-48a0-8064-18fc8df906b8","order_by":0,"name":"Narendra Kumar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABGUlEQVRIiWNgGAWjYNACNiDmgTB5+NkbGBgewIWJ0CIj2XOAgSGBFC02BjcS8GsxZz/78HFB2T15Bp4zpht+7mHgMbj5xvBDAoOdPINEWgI2LZY96cbGM84VGzbw9pjd7HnGwCN5O8dYIoEh2bBBIu0ANi0GB9LYpHnbEhgb+HnMbvAcYODhu51jBnQYcwKDRHoDVi3nn7H/BmqxB2m5+QeoheHmGZCWetxabqSxMQO1JIIcdhtki8ANHpCWw0AtOBx24xmzNM+5hOQ2nmNlt2UOSPBI9qQVSyQYHDds43mG1fsG59MYP/OUJdj28yRvu/nmgI09P/vhjR8+VFTL87OnGWDTAgfQOJCAGYU/IkfBKBgFo2AU4AcAWHBVtUd8ufMAAAAASUVORK5CYII=","orcid":"","institution":"University College London","correspondingAuthor":true,"submittingAuthor":false,"prefix":"","firstName":"Narendra","middleName":"","lastName":"Kumar","suffix":""},{"id":45573673,"identity":"efb32d4e-7223-4614-af9e-78aa4293180a","order_by":1,"name":"Eqram Rahman","email":"","orcid":"","institution":"University College London","correspondingAuthor":false,"submittingAuthor":false,"prefix":"","firstName":"Eqram","middleName":"","lastName":"Rahman","suffix":""}],"badges":[],"createdAt":"2021-07-04 04:59:05","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-685351/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-685351/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":17898330,"identity":"4eba1b7b-d82a-403f-89ea-66f903a15798","added_by":"auto","created_at":"2022-02-03 08:44:17","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":573983,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-685351/v1/62bcf348-72cc-4d9c-9244-717b5d53c54b.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Design and Validation of an integrated Objective Structured Clinical Examination (i-OSCE) for non-surgical aesthetics program","fulltext":[{"header":"Introduction","content":"\u003cp\u003eAssessment and evaluation are critical steps in medical education and rely on selecting a proper and robust instrument. Appropriate assessment tool helps to determine the effectiveness of educational programmes and ensures that the future clinicians are competent and suitable for independent clinical practice. However, the currently used assessment tools are insufficient to test the learners\u0026rsquo; knowledge, skills, behaviour, and critical thinking abilities holistically. In such cases, using the \u0026lsquo;test battery\u0026rsquo; approach becomes more practical to use a mix of assessment tools for measuring an array of learning domains (\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eTraditionally, clinical assessment strategies comprise a combination of \u0026lsquo;short\u0026rsquo; and \u0026rsquo;long case\u0026rsquo; evaluations. However, criticism about its low reliability (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e) and modern-day constraints such as increased litigation and student appeals (\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e) have led institutions to focus on exams that produce trustworthy, more easily defendable outcomes. Accordingly, conventional assessment strategy evolved to overcome the challenges of traditional methodologies, such as reliance on the patient\u0026rsquo;s performance, the examiner\u0026rsquo;s bias, the non-standardised grading scheme, and the candidate\u0026rsquo;s actual performance; the assessment strategy went through an evolution process. Consequently, the assessment process became standard, and the number of variable affective students\u0026rsquo; performance was reduced and paved the path favouring the introduction of objective structured clinical examination (OSCE), the \u0026ldquo;gold standard\u0026rdquo; for clinical assessments globally. It aims to examine the skills and ability to understand that the assessment results will reflect the trainee physicians\u0026rsquo; day-to-day clinical performance in real-life scenarios.\u003c/p\u003e \u003cp\u003eIn various research studies, OSCEs have been helpful in terms of reliability and validity(\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e). However, the long examination time is a cause of concern for the trainee physicians and costs to the program directors (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). OSCE mainly focuses on assessing affective, cognitive, and psychomotor learning domains. Nonetheless, performance is affected by various other factors such as knowledge to apply in real-life scenarios; non-clinical skills (decision-making, teamwork, resource management, planning, and critical thinking); attitudes; environment; emotional state; physical state; and personality traits. The drawback of OSCE is that it cannot be easily used to measure non-clinical skills (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e)\u003c/p\u003e \u003cp\u003eIn clinical education, critical thinking skills are measured by high-fidelity patient simulations, \u0026ldquo;California critical thinking skills test, California critical thinking disposition inventory, Del Bueno\u0026rsquo;s performance-based development system, health science reasoning test\u0026rdquo;, and Watson-Glaser critical thinking appraisal. However, these are limited in their lack of measuring the particular aspect of health profession-related attributes, inability to evaluate medical professionals\u0026rsquo; practical reality, and effectively assess psychometric properties (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eCritical appraisal is a subcategory of critical thinking, which specifies the ability to make clinical decisions by research evidence. Various studies have concluded that critical thinking can be refined, and without this essential ability, there can be drastically negative ramifications on trainee physicians\u0026rsquo; decisions. There has been evidence of a direct correlation between critical thinking and academic success; unfortunately, many trainee physicians struggle on tests explicitly measuring it (\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e). While evaluating critical thinking strategies, there is a prominent spotlight on evidence-based practice and its role in education. Numerous systematic reviews have inferred that clinically integrated assessment methods are needed to improve further evidence-based practice skills (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e). Critical appraisal has been incorporated into some of the high-stake professional and fellowship examinations where physicians are tested to assess their ability to judge a clinical paper in a short time based on its research design, result and whether to consider this to change one\u0026rsquo;s clinical practice.\u003c/p\u003e \u003cp\u003eHowever, there is no evidence in the literature reporting the development and implementation of an evaluation tool for assessing clinical skills, analytical thinking, and non-clinical skills in the NSA educational program. Therefore, the current study aims to develop and validate an integrated objective structured clinical examination (i-OSCE) by integrating clinical and critical thinking stations for the NSA postgraduate program.\u003c/p\u003e"},{"header":"Methods","content":"\u003ch4\u003eOSCE Station Blueprinting\u003c/h4\u003e\n\u003cp\u003eBlueprinting is the standard process of mapping the intended learning outcomes, which comprise knowledge and understanding, intellectual, practical, affective, and psychomotor skills relating to the postgraduate curriculum on NSA, with the knowledge and skill competencies to be tested in individual stations. An \u0026lsquo;expert panel\u0026rsquo; was formed consisting of five aesthetic practitioners and a senior clinical academic with over fifteen years of experience. A consensual and ceaseless approach was adopted to identify the tasks to be assessed, which are essential and relevant to the NSA practice, thereby validating the content of i-OSCE (Table 1).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eTable 1: OSCE Station Blueprinting\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://myfiles.space/user_files/58854_b38fc7f3db2c487f/58854_custom_files/img1629162764.png\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003ch4\u003eContent development and Validation\u003c/h4\u003e\n\u003cp\u003eA 2-day OSCE writing workshop was conducted for 15 aesthetic practitioners and clinical academicians in the presence of 3 expert facilitators, divided into three small groups. After a brief and structured presentation on OSCE, all the groups had a facilitator lead practice session to construct each station\u0026rsquo;s case scenarios. It was followed by critical feedback from the facilitator to the participants. The three working groups met regularly to construct case scenarios, candidate instruction, standardised patient information sheet, and, most importantly, the marking sheet where the entire scenario was deconstructed to make a performance checklist effectively to match the blueprint theme.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFinally, the expert panel was reconvened again to review the constructed cases with the checklists and parity of competencies across the cases. Fifty stations (25 Clinical and 25 Critical thinking) were selected to store in the repository managed by an OSCE administrator.\u0026nbsp;\u003c/p\u003e\n\u003ch4\u003eClinical Stations\u003c/h4\u003e\n\u003cp\u003eThese stations consisted of a simulated scenario of consultation in facial aesthetics. Here, candidates must take an appropriate history, clinical photography, and facial assessment to reach a specific, accurate diagnosis. Either standardised patients or patient actors were utilised within these stations. Any clinical examination skills relevant to facial aesthetics were subsidised to fit the station\u0026rsquo;s time limit. Candidates were given a brief history and asked to perform (either all or some aspects) a clinical examination and discuss it with the patient. At the end of the station, candidates were subsequently asked to summarise their findings or provide a brief management plan, including its justification, to the examiner. The other stations developed would assess candidates\u0026rsquo; professionalism and communication skills.\u003c/p\u003e\n\u003ch4\u003eCritical Thinking (CT) Stations\u003c/h4\u003e\n\u003cp\u003eTwenty-five CT stations were created, where candidates were asked to critically appraise its validity and reliability (formulating PICO, review methodology and critical analysis of the discussion), whether the article published in a peer-reviewed journal and decide to adopt this into clinical practice (applicability).\u003c/p\u003e\n\u003ch4\u003eMarking scheme\u0026nbsp;\u003c/h4\u003e\n\u003cp\u003eEvery separate checklist score was weighed based on the allotted task\u0026rsquo;s significance as the station author deemed, which later reached further agreement from an expert panel in a station review meeting. Finally, each station received an independent standardisation to create the pass marks with the help of the borderline regression method, which utilised a combination of the checklist score and the examiner\u0026rsquo;s single 3 points global rating (clear pass, borderline, or clear fail) (Table 2).\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eTable 2: Example of a checklist/marking scheme\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://myfiles.space/user_files/58854_b38fc7f3db2c487f/58854_custom_files/img1629162812.png\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003ch4\u003eAssessor Training\u003c/h4\u003e\n\u003cp\u003eAll the examiners participated in an hour-long orientation program to familiarise themselves with the OSCE setting, competency testing, and scoring guidelines. Further, they were provided with a guide describing the definitions of the competencies, checklist, global ratings.\u0026nbsp;\u003c/p\u003e\n\u003ch4\u003eStandardised Patient (SP) Training\u003c/h4\u003e\n\u003cp\u003eHealthy volunteers were recruited with the help of a modelling agency to act as \u0026lsquo;simulated\u0026rsquo; patients for all the stations. They went through coaching conducted by professional medical actors and clinical and communication skills experts. SPs were given a task, particularly to the station, and they practised until they played their roles consistently. As they were also responsible for completing their part of the checklist, a calibration video was shown to practice marking and debriefing.\u003c/p\u003e\n\u003ch4\u003ePilot Study\u003c/h4\u003e\n\u003cp\u003eA pilot study was conducted with ten aesthetic practitioners to examine the feasibility of i-OSCE. Planned eight stations were run with one examiner and one observer. Finally, the results were reviewed, including the feedback from individual stations to amend i-OSCE documents for final implementation during the summative examination.\u0026nbsp;\u003c/p\u003e\n\u003ch4\u003eFinal i-OSCE\u003c/h4\u003e\n\u003cp\u003eThe final examination was conducted comprised four clinical and four critical thinking stations of 15 minutes each and four rest stations with forty trainee physicians. The total run for the exam was 180 minutes.\u0026nbsp;\u003c/p\u003e\n\u003ch4\u003eStatistical Analysis\u003c/h4\u003e\n\u003cp\u003eFor an assessment tool to be accepted as reliable and valid, the most widely used statistical measurement is Cronbach\u0026rsquo;s alpha (11). However, some studies argue that it should not be used for internal reliability as sole measurement, as it is directly proportionate to the examination length; therefore, it indicates the station\u0026rsquo;s stability, not the internal consistency (12,13). However, concurrent use of Pearson correlation coefficient (PCC) or Spearman\u0026rsquo;s rank correlation helps overcome the issue (14). Therefore, PCC was used to investigate the strength of correlation between utilising the checklist and the global rating (clear pass, borderline, or clear fail), which helped provide a measure of the validity of the marking criteria used. For calculating the interrater reliability (IRR), Cronbach\u0026rsquo;s alpha was used through two-way mixed effects; intra-class correlations (ICC) for consistency and internal reliability. For interpretation of ICCs, Cicchetti\u0026rsquo;s classification (IRR less than 0.40 is poor; 0.40\u0026ndash;0.59 is fair; 0.60\u0026ndash;0.74 is good; 0.75\u0026ndash;1.00 is excellent) was used (15). Moreover, content validity was measured with the help of experts. Statistical analysis was carried out by using IBM SPSS Statistics for Mac, Version 27.0 (IBM Corp. Armonk, NY, USA).\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eThe Interclass correlation coefficient between the station checklist items was 0.946 (average measure upper bound 0.916, lower bound 0.968; p\u0026lt;0.00), considered to be significant (Table 3). The Inter-Item Correlation Matrix among the clinical station checklist items and critical thinking checklists also showed statistical significance (Table 4). The Pearson correlation coefficient (PCC) used to ascertain the correlation between checklist rating and global rating (Table 5), yielding a high correlation (0.80 to 0.934).\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTable 3: Intraclass Correlation Coefficient\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://myfiles.space/user_files/58854_b38fc7f3db2c487f/58854_custom_files/img1629162872.png\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eTable 4: Inter-Item Correlation Matrix of station checklist\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"https://myfiles.space/user_files/58854_b38fc7f3db2c487f/58854_custom_files/img1629162894.png\"\u003e\u003c/p\u003e\n\u003cp\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp style='margin-top:0in;margin-right:0in;margin-bottom:10.0pt;margin-left:0in;text-align:center;line-height:200%;font-size:12px;font-family:\"Calibri\",sans-serif;color:#1F497D;font-style:italic;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family:\"Arial\",sans-serif;color:black;font-style:normal;'\u003eTable 5: Checklist Vs Global Assessment Correlation\u003c/span\u003e\u003c/p\u003e\n\u003cdiv align=\"center\" style='margin-top:0in;margin-right:0in;margin-bottom:10.0pt;margin-left:0in;line-height:115%;font-size:15px;font-family:\"Calibri\",sans-serif;'\u003e\n \u003ctable style=\"border: none;width:207.0pt;border-collapse:collapse;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:29.4pt;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cstrong\u003e\u003cspan style='font-size:16px;line-height:200%;font-family:\"Arial\",sans-serif;color:black;'\u003eStation\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border:solid windowtext 1.0pt;border-left: none;padding:0in 5.4pt 0in 5.4pt;height:29.4pt;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cstrong\u003e\u003cspan style='font-size:16px;line-height:200%;font-family:\"Arial\",sans-serif;color:black;'\u003ePearson Correlation\u003c/span\u003e\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e1\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.865\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e2\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.82\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e3\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.934\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e4\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.8\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e5\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.856\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e6\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.848\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e7\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.891\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width:71.0pt;border:solid windowtext 1.0pt;border-top: none;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e8\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width:136.0pt;border-top:none;border-left:none;border-bottom:solid windowtext 1.0pt;border-right:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:.3in;\"\u003e\n \u003cp style='margin-top:0in;margin-right:0in;margin-bottom:0in;margin-left:0in;line-height: 200%;font-size:15px;font-family:\"Calibri\",sans-serif;text-align:justify;'\u003e\u003cspan style='font-size:16px;line-height:200%;font-family: \"Arial\",sans-serif;color:black;'\u003e0.875\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis is the first integrated OSCE validation study combining clinical and critical thinking skills for a postgraduate NSA education to the best of the author\u0026rsquo;s knowledge. OSCE is a flexible assessment method use to evaluate competence by direct observation based on objective assessment criteria. It is composed of several \u0026ldquo;stations\u0026rdquo; where examinees are required to conduct a range of clinical tasks against required clinical competence displaying the skills and attitudes over a given duration. The OSCE has been used to assess the skills most important to the healthcare professionals\u0026rsquo; success, such as data acquisition, interpretation, troubleshooting, engagement, and management of erratic patient behaviour, otherwise difficult to obtain during the classic clinical review (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). Miller\u0026rsquo;s framework for clinical competency development recommended four stages; \u0026ldquo;knows the facts\u0026rdquo;; \u0026ldquo;knows how to elaborate and integrate the understanding\u0026rdquo;; \u0026ldquo;shows how\u0026rdquo; they apply knowledge, skills and attitude for the patient outcome; and finally \u0026ldquo;does\u0026rdquo; employ all the skills in their independent practice to serve the community, proven to work reasonably well in medical education settings (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e). Evidence suggests that the OSCE helps assess the third stage \u0026ldquo;shows how\u0026rdquo; by concentrating on the clinical skills in a safe learning environment.\u003c/p\u003e \u003cp\u003eCritical thinking is considered to be a crucial cognitive method for the creation and utilisation of knowledge. It plays a functional role during problem-solving and decision-making in a social, clinical, or ethical context. Moreover, it is equally valuable for analysing complex data, assessing situations, and implementing the most suitable actions. In a recent article, \u0026ldquo;critical thinking is described as a cognitive process, purposeful, self-regulatory judgment that has two components of cognitive skills (interpretation, analysis, inference, evaluation, explanation, and self-regulation) and a motivational component (the disposition toward critical thinking)\u0026rdquo; (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eMore focus has been put in recent years on improving higher-level thought (critical thinking and clinical reasoning) skills to help physicians retain clinical integrity and medical professionalism. More than two-thirds of the reported mistakes in diagnosis are linked to physician\u0026rsquo;s lack of critical thinking ability in the present context. Given the belief that healthcare professionals must be logical thinkers, there is no consensus on the most successful model to teach and evaluate critical thinking and clinical reasoning skills (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e). Recent research, which evaluated a wide range of quantitative and qualitative competencies, including behavioural and communication skills, showed that the OSCE was valid and reliable and essential for positive educational effects. Several authors have advocated that emphasising an aim of OSCEs is to develop affability in critical thinking as a precursor to practising (\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eThere is no valid assessment instrument combining clinical and critical appraisal skills to evaluate safe practice in non-surgical aesthetics. Therefore, using different stations to evaluate clinical skills and critical thinking ability is beneficial in this evaluation. The various clinical skills measured were consultation skills pertaining to the NSA, knowledge of the signs of ageing and the underlying anatomy, assessment of skin quality, full-face assessment to identify treatment needs for optimal results, clinical photography, development of an efficient and optimal treatment plan, safer injection techniques, post-treatment advice, complication management, and situational judgments. During the evaluation, the critical appraisal skills measured included understanding the research relating to facial assessment, botulinum toxin science, rheology of the soft-tissue fillers and complication management.\u003c/p\u003e \u003cp\u003eIt has shown that the generalisation coefficients appear to differ significantly from 0.40 to 0.85, while the majority of these coefficients vary from 0.5\u0026ndash;0.6 (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). In the present study, the average intraclass correlation coefficient measures range between 0.916 to 0.968, which is more than the reliability coefficient threshold of 0.8 or over. The variability in the generalizability coefficients may be attributed to the examinees\u0026rsquo; variable performance on different OSCE stations (content specificity). I-OSCE is shown to be robust and able to test applicants for their competence to carry out multiple component tasks.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIntegrated OSCE has demonstrated to be a reliable and accurate assessment tool for examining the trainee aesthetic physicians\u0026rsquo; professional competence. This tool has objectively evaluated trainee physicians critical thinking and clinical skills, including clinical reasoning. The program directors should consider the deployment of i-OSCE along with OSPE as an assessment tool in the postgraduate curriculum for non-surgical aesthetics.\u003c/p\u003e"},{"header":"Abbreviations","content":"\u003cdiv class=\"DefinitionList\"\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eNSA\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003enon-surgical aesthetics\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eOSCE\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eObjective structured clinical examination\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eI-OSCE\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eIntegrated Objective structured clinical examination\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eOSPE\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eObjective structured practical examination\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003cdiv class=\"DefinitionListEntry\"\u003e \u003cdiv class=\"Term\"\u003eSP\u003c/div\u003e \u003cdiv class=\"Description\"\u003e \u003cp\u003eStandardised patient\u003c/p\u003e \u003c/div\u003e \u003c/div\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics Approval and Consent to Participate\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe study was approved by the Faculty Research Ethics Panel of the Faculty of Medical Science, Anglia Ruskin University, Chelmsford, UK.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u0026nbsp;\u003c/strong\u003eNot applicable\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials:\u003c/strong\u003e The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interest:\u003c/strong\u003e The authors declare no conflict of interest.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u0026nbsp;\u003c/strong\u003eNo funding.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor\u0026rsquo;s contribution:\u0026nbsp;\u003c/strong\u003eNK drafted and prepared the manuscript. ER supervised and edited the manuscript. NK and ER analysed and interpreted the data and approved the final manuscript.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements:\u0026nbsp;\u003c/strong\u003eThe authors would like to thank all the experts who participated in during the blueprinting process and subsequently acted as examiners. Authors would also like to thank actors who performed the role of standard patients; without their support this study would have not been possible.\u003c/p\u003e"},{"header":"References","content":"\u003cp\u003e1. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Tabish SA. Assessment methods in medical education. Int J Health Sci (Qassim) [Internet]. 2008 Jul;2(2):3\u0026ndash;7. Available from: http://www.ncbi.nlm.nih.gov/pubmed/21475483\u003c/p\u003e\n\u003cp\u003e2. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Khan KZ, Gaunt K, Ramachandran S, Pushkar P. The Objective Structured Clinical Examination (OSCE): AMEE Guide No. 81. Part II: Organisation \u0026amp;amp; Administration. Med Teach [Internet]. 2013 Sep 22;35(9):e1447\u0026ndash;63. Available from: http://www.tandfonline.com/doi/full/10.3109/0142159X.2013.818635\u003c/p\u003e\n\u003cp\u003e3. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Tey C, Chiavaroli N, Ryan A. Perceived educational impact of the medical student long case: a qualitative study. BMC Med Educ [Internet]. 2020 Dec 7;20(1):257. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-020-02182-6\u003c/p\u003e\n\u003cp\u003e4. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Pell G, Roberts TE. Setting standards for student assessment. Int J Res Method Educ [Internet]. 2006 Apr;29(1):91\u0026ndash;103. Available from: http://www.tandfonline.com/doi/abs/10.1080/01406720500537486\u003c/p\u003e\n\u003cp\u003e5. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Kumar N, Bhardwaj S, Rahman E. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom : a cohort study. Adv Med Educ Pract. 2018;9:239\u0026ndash;45.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e6. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Marwaha S. Objective Structured Clinical Examinations (OSCEs), psychiatry and the Clinical assessment of Skills and Competencies (CASC)Same Evidence, Different Judgement. BMC Psychiatry [Internet]. 2011 Dec 16;11(1):85. Available from: http://bmcpsychiatry.biomedcentral.com/articles/10.1186/1471-244X-11-85\u003c/p\u003e\n\u003cp\u003e7. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Nguyen K, Ben Khallouq B, Schuster A, Beevers C, Dil N, Kay D, et al. Developing a tool for observing group critical thinking skills in first-year medical students: a pilot study using physiology-based, high-fidelity patient simulations. Adv Physiol Educ [Internet]. 2017 Dec 1;41(4):604\u0026ndash;11. Available from: https://www.physiology.org/doi/10.1152/advan.00126.2017\u003c/p\u003e\n\u003cp\u003e8. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Majumder MAA, Sa B, Alateeq FA, Rahman S. Teaching and Assessing Critical Thinking and Clinical Reasoning Skills in Medical Education. In 2019. p. 213\u0026ndash;33. Available from: http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-5225-7829-1.ch012\u003c/p\u003e\n\u003cp\u003e9. \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;West DC, Pomeroy JR, Park JK, Gerstenberger EA, Sandoval J. Critical Thinking in Graduate Medical Education. JAMA [Internet]. 2000 Sep 6;284(9):1105. Available from: http://jama.jamanetwork.com/article.aspx?doi=10.1001/jama.284.9.1105\u003c/p\u003e\n\u003cp\u003e10. \u0026nbsp; \u0026nbsp;\u0026nbsp;Maudsley G, Strivens J. Promoting professional knowledge, experiential learning and critical thinking for medical students. Med Educ. 2000 Jul;34(7):535\u0026ndash;44.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e11. \u0026nbsp; \u0026nbsp;\u0026nbsp;Swanson DB, Stillman PL. Use of Standardized Patients for Teaching And Assessing Clinical Skills. Eval Health Prof [Internet]. 1990 Mar 29;13(1):79\u0026ndash;103. Available from: http://journals.sagepub.com/doi/10.1177/016327879001300105\u003c/p\u003e\n\u003cp\u003e12. \u0026nbsp; \u0026nbsp;\u0026nbsp;Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika [Internet]. 1951 Sep;16(3):297\u0026ndash;334. Available from: http://link.springer.com/10.1007/BF02310555\u003c/p\u003e\n\u003cp\u003e13. \u0026nbsp; \u0026nbsp;\u0026nbsp;Streiner DL. Starting at the Beginning: An Introduction to Coefficient Alpha and Internal Consistency. J Pers Assess [Internet]. 2003 Feb;80(1):99\u0026ndash;103. Available from: http://www.tandfonline.com/doi/abs/10.1207/S15327752JPA8001_18\u003c/p\u003e\n\u003cp\u003e14. \u0026nbsp; \u0026nbsp;\u0026nbsp;Graham JM. Congeneric and (Essentially) Tau-Equivalent Estimates of Score Reliability. Educ Psychol Meas [Internet]. 2006 Dec 2;66(6):930\u0026ndash;44. Available from: http://journals.sagepub.com/doi/10.1177/0013164406288165\u003c/p\u003e\n\u003cp\u003e15. \u0026nbsp; \u0026nbsp;\u0026nbsp;Al-Osail AM, Al-Sheikh MH, Al-Osail EM, Al-Ghamdi MA, Al-Hawas AM, Al-Bahussain AS, et al. Is Cronbach\u0026rsquo;s alpha sufficient for assessing the reliability of the OSCE for an internal medicine course? BMC Res Notes [Internet]. 2015 Dec 19;8(1):582. Available from: http://www.biomedcentral.com/1756-0500/8/582\u003c/p\u003e\n\u003cp\u003e16. \u0026nbsp; \u0026nbsp;\u0026nbsp;Zayyan M. Objective Structured Clinical Examination: The Assessment of Choice. Oman Med J [Internet]. 2011 Jul 25;26(4):219\u0026ndash;22. Available from: http://www.omjournal.org/fultext_PDF.aspx?DetailsID=114\u0026amp;pdf=images/114_M_Deatials_Pdf_.pdf\u0026amp;type=pdf\u003c/p\u003e\n\u003cp\u003e17. \u0026nbsp; \u0026nbsp;\u0026nbsp;Miller GE. The assessment of clinical skills/competence/performance. Acad Med [Internet]. 1990 Sep;65(9):S63-7. Available from: http://journals.lww.com/00001888-199009000-00045\u003c/p\u003e\n\u003cp\u003e18. \u0026nbsp; \u0026nbsp;\u0026nbsp;Sousa AC, Wagner DP, Henry RC, Mavis BE. Better data for teachers, better data for learners, better patient care: college-wide assessment at Michigan State University\u0026rsquo;s College of Human Medicine. Med Educ Online [Internet]. 2011 Jan 14;16(1):5926. Available from: https://www.tandfonline.com/doi/full/10.3402/meo.v16i0.5926\u003c/p\u003e\n\u003cp\u003e19. \u0026nbsp; \u0026nbsp;\u0026nbsp;Nematbakhsh M, Sharif S-M, Athari Z-S, Nasr A. Assessing critical thinking in medical sciences students in two sequential semesters: Does it improve? J Educ Health Promot. 2013;2(1):5.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e20. \u0026nbsp; \u0026nbsp; Krusen NE, Rollins D. Design of an OSCE to Assess Clinical Competence of Occupational Therapy Students. J Occup Ther Educ [Internet]. 2019 Jan 1;3(1). Available from: https://encompass.eku.edu/jote/vol3/iss1/11\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"OSCE, Non-Surgical Aesthetics, Clinical Examination, Critical Thinking, Clinical Reasoning","lastPublishedDoi":"10.21203/rs.3.rs-685351/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-685351/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eIntroduction\u003c/strong\u003e: The Objective Structured Clinical Examination (OSCE) is a popular and practical method for evaluating trainee physicians’ competencies. To help assess non-surgical aesthetics students’ critical thinking and relevant abilities, we developed a novel assessment tool based on the OSCE, the i-OSCE (Integrated objective structured Clinical examination).\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eMethods\u003c/strong\u003e: Initially, an expert panel consisting of 5 Aesthetic Practitioners with over fifteen years of experience and a senior clinical academic were selected to develop a blueprint for i-OSCE. Through this blueprint, essential qualities and skills were identified for the assessment. To ensure the process standardisation, training workshops for examiners and simulated patients were organised. The final i-OSCE consisted of 12 stations (four clinical, four critical thinking, and four rest stations lasting 180 minutes.\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e The Interclass correlation coefficient between the station checklist items was 0.946 (average measure upper bound 0.916, lower bound 0.968; p \u0026lt; 0.00), considered to be significant. The Inter-Item Correlation Matrix among the clinical station checklist and critical thinking checklist items also showed statistical significance. The Pearson correlation coefficient (PCC) used to ascertain the correlation between checklist rating and global rating, yielding a high correlation (0.80 to 0.934).\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eConclusion\u003c/strong\u003e: The i-OSCE has been proven to be a useful and reliable assessment tool to evaluate clinical competence and critical thinking in non-surgical aesthetics education.\u003c/p\u003e","manuscriptTitle":"Design and Validation of an integrated Objective Structured Clinical Examination (i-OSCE) for non-surgical aesthetics program","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2021-08-17 14:54:20","doi":"10.21203/rs.3.rs-685351/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"cd35cb05-ba13-446a-a1d3-4e88a865e7a9","owner":[],"postedDate":"August 17th, 2021","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":6493229,"name":"Educational Philosophy and Theory"}],"tags":[],"updatedAt":"2022-02-03T08:44:08+00:00","versionOfRecord":[],"versionCreatedAt":"2021-08-17 14:54:20","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-685351","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-685351","identity":"rs-685351","version":["v1"]},"buildId":"-HB7Z8yhvgn0wM9Nzuekk","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}