Overcoming Challenges in Implementation of Competency- Based Medical Education in India: Leveraging Postgraduates for Multiple Choice Item Design for Undergraduate Assessments in Pathology

Overcoming Challenges in Implementation of Competency- Based Medical Education in India: Leveraging Postgraduates for Multiple Choice Item Design for Undergraduate Assessments in Pathology | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Overcoming Challenges in Implementation of Competency- Based Medical Education in India: Leveraging Postgraduates for Multiple Choice Item Design for Undergraduate Assessments in Pathology Reshma G. Kini, Nisha J Marla, Christol Blanch Moras This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-5867554/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Background The implementation of Competency-Based Medical Education (CBME) in India has highlighted significant challenges, including faculty shortages and the need for standardized assessments. Pathology postgraduates, often involved in undergraduate teaching, represent an underutilized resource. Training them in Multiple Choice Item (MCI) development and validation can improve assessment quality while addressing gaps in CBME implementation. Method A structured training program was conducted for nine first-year pathology postgraduates at a single institution. The program included nine interactive sessions on CBME principles, Bloom’s taxonomy, MCI construction, identification of technical flaws, validation processes, and blueprinting. Participants’ reactions and self-perceived learning were evaluated using Kirkpatrick Levels 1 and 2 through pre- and post-program surveys and structured feedback forms. Product evaluation analyzed the MCIs for difficulty and discrimination indices, and the need for revalidation by subject matter experts (SMEs). Open-ended feedback was thematically analyzed. Results Participants showed significant improvements in knowledge and confidence in MCI development (p < 0.05). Product evaluation revealed that 43.2% of MCIs required revalidation, but most met acceptable thresholds for difficulty (62.5%) and discrimination (75%). Feedback highlighted the program’s structured and practical focus, while suggesting improvements in session pacing and content depth. Multiple choice items predominantly targeted lower order thinking skills, reflecting participants’ novice status and the need for longitudinal support to improve question complexity. Conclusion This study demonstrates the feasibility of leveraging pathology postgraduates to address CBME implementation challenges. By equipping postgraduates with assessment design skills, the program improved undergraduate assessment quality while addressing faculty shortages. Future efforts should focus on expanding such programs across institutions, incorporating longitudinal mentorship, and evaluating their long-term impact on medical education outcomes. Competency-Based Education Pathology Postgraduate Education Multiple choice items Residents as teachers Figures Figure 1 Background Competency-Based Medical Education (CBME) represents a paradigm shift in medical training, emphasizing the achievement of specific competencies over time-based progression. In India, the National Medical Commission (NMC) has mandated the implementation of CBME in undergraduate medical education from 2019, introducing new challenges and demands for faculty development, curriculum design, and assessment standardization. With over 700 medical colleges in India, implementing CBME on such a large scale presents significant logistical and resource challenges. These include a shortage of trained faculty and a lack of standardization in assessment practices, particularly in regions with limited institutional resources.[1–3] To address these challenges, leveraging pathology postgraduates as educators represents an innovative and practical solution. Postgraduates, often tasked with supporting undergraduate teaching, are well-positioned to contribute meaningfully to the implementation of CBME if provided with appropriate training. Developing their skills in assessment design can not only enhance their own teaching competencies but also alleviate some of the faculty shortages that hinder CBME implementation. Specifically, training postgraduates in the construction and validation of Multiple-Choice Items (MCIs) can improve the quality of undergraduate assessments, ensuring alignment with curricular objectives and competency-based frameworks.[4,5] We use the framework of ‘Residents as Teachers’. In western countries structured programs have been published for training residents in teaching methodologies. In absence of such data from India we conducted a preliminary survey across 11 medical colleges running postgraduate pathology program(unpublished). The survey revealed while all participating institutions reported involving PGs in undergraduate teaching, only 35% provided structured teaching-methodology training and none involved them in undergraduate assessment.[5–7] This study aimed to evaluate the effectiveness of a structured training program designed to improve assessment literacy by enhancing pathology postgraduates' competencies in developing MCIs for undergraduate assessments. Using Kirkpatrick Levels 1 and 2 as the evaluation framework, the program assessed participants' reaction and learning outcomes. Additionally, an unconventional component—product evaluation—was incorporated to analyze the quality of the MCQs produced during the program. The broader goal was to explore how such training programs can address CBME implementation challenges by leveraging postgraduate educators and improving the standardization of assessment practices.[8,9] In this manuscript, we present the outcomes of the program, including participants' improvements in knowledge, confidence, and MCI development skills, alongside insights from product evaluation. The findings contribute to the growing discourse on innovative strategies for CBME implementation, particularly in CBME naïve regions experiencing resource challenges.[10,11] Methodology Study Design and Setting This was a prospective, interventional study conducted at a single institution in India. The study aimed to evaluate the effectiveness of a structured training program designed to enhance the competencies of first-year pathology postgraduates in developing Multiple Choice Items (MCIs) for undergraduate medical education. The program was conducted over a span of 3 weeks. Participants The study included nine first-year postgraduate students enrolled in the Department of Pathology. All participants provided informed consent prior to their inclusion in the program. Participants were selected based on their enrollment in the department and their involvement in teaching activities for undergraduate students. Program Structure The training program consisted of nine interactive sessions, each lasting for 1-1.5-hour duration The design of the sessions and outcomes is provided in Table 1 Table 1: The postgraduate training program with sequence and structure of individual sessions of individual Session Number Title/Topic Learning Objectives Key Content/Activities Mode of Delivery Duration Resources/Tools Used Assessment Component Output/Deliverable Key Outcomes 1 Introduction to CBME a - Understand the principles of CBME and its significance. Overview of CBME, its goals, and alignment with assessments. Interactive Lecture 1.5 hours Presentation Slides Pre-session quiz Understanding of CBME framework. Participants oriented to CBME principles. 2 Bloom's Taxonomy and MCIs b - Apply Bloom’s taxonomy to design questions assessing LOTS c and HOTS d . Explanation of cognitive levels and examples of MCIs. Workshop 1.5 hours Bloom’s taxonomy chart, examples Group activity on taxonomy alignment. Draft MCIs classified by Bloom’s levels. Improved ability to classify cognitive skills. 3 MCI Writing Principles - Learn guidelines for writing effective MCQs. Best practices for item writing and avoiding common flaws. Hands-on Session 1.5 hours Item-writing guidelines document Peer-reviewed MCI drafts Draft MCIs adhering to guidelines. Enhanced quality of drafted MCQIs. 4 Identifying Technical Flaws - Identify and eliminate technical flaws in MCQs. Examples of flawed questions and solutions. Group Discussion 1.5 hours Example question bank Peer review session Revised MCQs with fewer technical flaws. Participants learned to recognize flaws. 5 Validation of MCQs - Ensure content, face, and construct validity in MCQs. Steps in validating MCQs; expert feedback. Interactive Lecture 1.5 hours Validation checklist SME validation feedback Validated MCQs ready for use. Participants gained validation skills. 6 Difficulty and Discrimination - Analyze MCQs using difficulty and discrimination indices. Explanation of indices, examples, and calculations. Hands-on Workshop 1.5 hours Spreadsheet for index calculations Post-session group analysis MCQs with analyzed indices. Understanding of quality metrics for MCQs. 7 Blueprinting for Assessments - Develop assessment blueprints aligned with curricular objectives. Steps to create a blueprint and align with competencies. Workshop 1.5 hours Blueprint templates, curriculum docs Blueprint review Blueprinted assessments. Assessments aligned with objectives. 8 Peer and Expert Review - Incorporate peer and expert feedback to improve MCQs. Structured review process; feedback integration. Collaborative Session 1.5 hours Peer review rubric Feedback forms Revised MCQs incorporating feedback. Enhanced quality through iterative improvement. 9 Technology for Assessment - Use digital tools for creating, deploying, and analyzing MCQs. Demonstration of Google Forms and other tools. Interactive Demo 1.5 hours Google Forms, analysis software Post-session quiz Digitized MCQs for assessments. Participants equipped with tech skills. a-Competency Based Medical Education b- Multiple Choice Items c- Lower Order Thinking Skills d- Higher Order Thinking Skills Evaluation Framework The effectiveness of the training program was evaluated using Kirkpatrick’s model: Level 1: Reaction Participant satisfaction with the program was assessed through structured feedback forms administered at the end of the program. Feedback included Likert-scale ratings on session relevance, content, objectives, and duration, as well as open-ended questions to gather qualitative insights. Level 2: Learning- Surveys Participants’ improvements in knowledge and confidence were evaluated using a pre- and post-program survey. The survey consisted of 20 items addressing key competencies, such as understanding Bloom’s taxonomy, identifying technical flaws, and applying validation. A retro-prequestionnaire addressing their knowledge and confidence in the major aspects of training component was also taken. Level 2: Learning- Product Evaluation The MCIs created by participants during the program were subjected to peer validation. Following this the MCIs were evaluated by subject matter experts (SMEs). MCQs were assessed for validity. A set of 30 randomly selected MCQs which needed no validation were deployed as a test for undergraduate students. Difficulty indices, and discrimination indices was calculated from the responses. Statistical Analysis Quantitative data from the pre- and post-program surveys were analyzed using Wilcoxon Signed-Rank Test] to determine the significance of changes in knowledge and confidence. Descriptive statistics were used to summarize feedback responses. The quality of MCIs was analyzed based on the percentage of items meeting predefined thresholds for difficulty (0.31–0.70) and discrimination (≥0.30). The statistical analysis was done using SPSS software version 21. The open-ended feedback was analyzed qualitatively through thematic analysis. Responses collected via structured feedback forms were reviewed to identify recurring themes, highlighting positive aspects such as training usefulness, session clarity, and relevance to assessment. Suggestions for improvement, like session scheduling and content depth, were categorized. Key themes and illustrative quotes were summarized and incorporated into the results to provide qualitative insights into participants' reactions. Ethical Considerations The study was approved by the institutional ethics committee. Participants were informed about the purpose of the study and provided written consent. Confidentiality of participant responses and MCIs was maintained throughout the study. Results Participant Demographics Nine first-year pathology postgraduate students participated in the study, attending all nine sessions of the training program. Their participation rate was 100%, ensuring consistency in feedback and evaluation. Kirkpatrick Level 1: Reaction Participants' reactions were captured using a 5-point Likert scale questionnaire and open-ended feedback. Responses evaluated content appropriateness, session objectives, relevance, session timing, and overall enjoyment. (Fig 1) Open-ended responses to the question "What was good about the sessions?" were thematically analyzed. Six key themes emerged: Awareness of Complexity in MCQ Construction : Participants appreciated learning about the multiple factors involved in framing MCQs. "I realized that there is so much behind preparing MCQs only after attending these sessions." Knowledge Enhancement and New Insights : The sessions introduced new concepts and expanded participants' understanding. "Learnt new concepts which were fascinating." Practical and Relevant Examples : The use of real-world examples was highlighted as particularly beneficial. "Relevant examples on how to improve MCQ formation." Clarity and Structured Delivery : Participants praised the clarity and structured approach of the sessions. "Explained in detail from the scratch." Applicability to Broader Contexts : The knowledge gained was seen as relevant beyond the immediate training. "It broadened my knowledge on how parallel entrance coaching works." Informative and Engaging Content : Participants described the sessions as informative and engaging. "The sessions were informative, and a different aspect of learning and question preparation was learnt." This thematic analysis demonstrates the effectiveness of the training in meeting its objectives while providing practical, engaging, and insightful learning experiences. Participant feedback on potential improvements highlighted the following themes: Session Scheduling : Continuous, back-to-back sessions were found to be exhausting, with suggestions for more spaced-out sessions and fixed goals. "Preferably spaced-out sessions. And fixed goals for each class would have been ideal." Time Management : Improved allocation of time for individual sessions was recommended. "Time management." Depth of Content : Participants expressed interest in more detailed discussions on question-framing methods. "Few more sessions on question-framing methods might be of use." This feedback reflects a desire for optimized scheduling and deeper exploration of advanced topics to enhance the learning experience. Self-Perceived Changes Participants' perceived changes in knowledge and confidence were evaluated using a retro-prequestionnaire. The results are presented in Table 2. The results demonstrate that the training program had a meaningful and positive impact on the participants’ understanding of key educational concepts and their confidence in implementing these skills. The statistical significance supports the effectiveness of the intervention. Table 2. Self-Perceived Knowledge and Confidence Knowledge on Learning Domains before vs after session Knowledge regarding Blooms taxonomy before vs after the sessions Knowledge HOTS c before vs after the sessions Confidence in making MCQs for HOTS and LOTS d before vs after the sessions Knowledge on concept of validity before vs after the sessions Confidence to perform post-validation before vs after the sessions Z -2.714 b -2.598 b -2.555 b -2.565 b -2.565 b -2.558 b Asymp. Sig. (2-tailed) .007 .009 .011 .010 .010 .011 a. Wilcoxon Signed Ranks Test b. Based on negative ranks. c . HOTS- Higher Order Thinking Skills d LOTS- Lower Order Thinking Skills Significance at p <0.05 Participants created 340 MCIs, in topics commonly taught to undergraduates. The which were peer-reviewed and also assessed by subject matter experts (SMEs). Revalidation : 43.2% of MCIs required SME revalidation after peer review. Reasons included issues with: Face validity (26%), Content validity (15%), Construct validity (69%). Item Metrics : Difficulty Index : Acceptable in (213) 62.5%. Easy in 122 (35.8) % Discrimination Index : Good/Excellent in 75%. Discussion This study evaluated a structured training program designed to enhance pathology postgraduates' abilities in developing Multiple Choice Items (MCIs) for undergraduate assessments, utilizing Kirkpatrick's Levels 1 and 2 model of training evaluation. The findings indicate significant improvements in participants' knowledge, confidence, and skills related to MCI construction, aligning with educational theories and corroborating existing literature.[8] Leveraging Postgraduates for CBME Implementation The findings underscore the feasibility of leveraging postgraduates as educators to address challenges in implementing CBME. As of January 2025, India has approximately 779 medical colleges. Implementing Competency-Based Medical Education (CBME) across such a vast number of institutions presents significant challenges, including faculty development, infrastructure enhancement, and standardization of assessment methods. In this context, training postgraduate students in assessment methodologies could be invaluable. However, it is essential to consider their readiness and ensure they receive adequate support to perform effectively in these roles..[4–6] Also, the program addresses a critical gap identified in prior needs assessment surveys, where many institutions employed postgraduates in teaching roles without formal training. This initiative fulfills an institutional obligation to provide postgraduates with the competencies needed to design robust, valid assessment tools The ability to design and validate Multiple Choice Items (MCIs) is a valuable skill, particularly with the increasing reliance on standardized assessments like the National Exit Test. Furthermore, introducing PGs to robust assessment practices early in their careers promotes lifelong skills in education and evaluation, and enhance their learning of pathology. [7,12–17] Several studies have concluded that training faculty in developing high quality MCQs is effective. The uniqueness of our program is that we aimed at enhancement of assessment literacy among first year postgraduates. Our program included components of blueprinting, using digital tools such as google forms for assessment and fundamental concepts of Bloom’s taxonomy, Competency- based education, deriving Specific learning objective. And MCQ validation. This was aimed at holistic development of a postgraduate as an assessor of undergraduates. However the focus of our evaluation was on the quality of the MCIs produced.[18–20] The feasibility of conducting this program was because the sessions could be conducted at the department by the department faculty, and it was of short duration. This is an important consideration when integrating the program in the current postgraduate curriculum. Insights from Participant Reactions and Learning Outcomes The effectiveness of the program is evidenced in multiple ways. The improvements observed in participants' knowledge and confidence in creating MCIs are consistent with foundational educational theories such as Kolb’s experiential learning Theory emphasizes learning through a cycle of experience, reflection, conceptualization, and experimentation. The program’s emphasis on practical tasks, such as creating and validating MCIs, enabled participants to apply theoretical principles in real-world scenarios while refining their skills based on peer and expert feedback. The secondary benefit of deeper understanding of the subject and enhancing self-learning could be deduced by the quality of the MCIs produced.[18,21] We focused the study on evaluating the collective impact of the training program on the participants as a group, hence we did not analyse individual postgraduate’s performances Given the relatively homogenous background of the participants—all being first-year pathology postgraduates—individual-level analysis was not expected to yield additional insights beyond those provided by group-level trends. Moreover, with a small cohort size, categorizing individual performances might have limited statistical utility and could detract from the broader implications of the findings. Product Evaluation and Quality of MCIs Insights from the product evaluation highlight that a significant percentage of MCQs required revalidation by subject matter experts (SMEs). Furthermore, the majority of the MCQs produced were focused on assessing lower order thinking skills, predominantly at the "understanding" level of Bloom's taxonomy. Several factors could account for these findings. First, all participants were first-year postgraduates, and their limited in-depth knowledge of pathology may have constrained their ability to generate high-quality MCQs that assess higher-order thinking skills. Second, the absence of a longitudinal program with ongoing mentorship and support may have hindered their ability to refine and improve their MCQs over time. Finally, the SMEs involved in the validation process were highly experienced, potentially creating a substantial gap in expertise between the two groups. This disparity could have contributed to the higher revalidation rates observed during the evaluation.[22] In contrast to this majority of the MCQs had acceptable difficulty and discriminatory Indices making our findings comparable to those of Pham et al and Janzen et alwho compared the student and faculty generated MCIs.[23,24] This could be a promising trend as leveraging students in generating MCQs could overcome the challenge of generating question banks for continuous assessments mandated by the NMC[23,25–27]. However, certain limitations must be acknowledged. The small sample size and single-institution setting may limit the generalizability of the findings. Future research should include larger, more diverse cohorts across multiple institutions to strengthen external validity. Additionally, while this study focused on immediate improvements in knowledge and confidence, it did not evaluate the long-term retention of skills or their application in actual educational settings. Longitudinal studies are recommended to assess the sustained impact of such training programs. Finally, the study was specific to pathology postgraduates, and exploring the effectiveness of similar programs in other medical specialties could provide valuable insights into the broader applicability of the training model[28]. Implications for Medical Education This study demonstrates the potential of structured training programs to enhance postgraduate competencies in assessment design, contributing to the broader goals of CBME. The findings highlight the importance of incorporating product evaluation as an additional dimension in program evaluation, bridging the gap between learning outcomes and real-world application. Furthermore, the scalability of such programs across institutions and specialties could address systemic challenges in medical education, particularly in resource-constrained settings. Conclusion In conclusion, this study highlights the feasibility and effectiveness of a structured training program in enhancing postgraduates' skills in MCI development. By aligning with the NMC’s mandate and integrating evidence-based educational practices, the program demonstrates its potential to improve postgraduate teaching competencies and undergraduate assessment quality, contributing to the broader goals of competency-based medical education. Declarations Ethics approval was obtained institutional ethics committee and consent to participate was obtained from the participants Consent for publication- not applicable Availability of data and materials- The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request. Competing interests- authors declare no competing intrests Funding- the study was self-funded Authors' contributions- All the authors contributed to the article as per the authorship criteria Acknowledgements: nil Clinical trial number: not applicable References NMC, National Exit Test Regulations, 2023 | NMC [Internet]. [cited 2025 Jan 17]. Available from: https://www.nmc.org.in/rules-regulations/nmc-national-exit-test-regulations-2023/ PG Curricula | NMC [Internet]. [cited 2025 Jan 17]. Available from: https://www.nmc.org.in/information-desk/for-colleges/pg-curricula-2/ UG Curriculum | NMC [Internet]. [cited 2024 Nov 17]. Available from: https://www.nmc.org.in/information-desk/for-colleges/ug-curriculum/ Kurien M, Thomas K. Competency Based Medical Education (CBME) in India: Clinicians’ Multifaceted Challenges. Indian J Otolaryngol Head Neck Surg [Internet]. 2024 Aug 1 [cited 2025 Jan 18];76(4):3730–4. Available from: https://doi.org/10.1007/s12070-024-04675-y McKinney CM, Hart R, Patterson AC, McKinney CM, Hart R, Patterson AC. Beyond Residents-as-Teachers: The Development of an Advanced Medical Education Pilot Elective for Pediatric Residents. Cureus [Internet]. 2023 Jun 25 [cited 2025 Jan 17];15(6). Available from: https://www.cureus.com/articles/161105-beyond-residents-as-teachers-the-development-of-an-advanced-medical-education-pilot-elective-for-pediatric-residents Ismail MA, Hamed OAE, Ismail SA. Residents as Professionalism Teachers and Assessors; A Pilot Study on Implementing a Faculty Development Program Leading to Curriculum Development. Int J Contemp Res Rev [Internet]. 2017 Oct 15 [cited 2025 Jan 18];8(10). Available from: http://ijcrr.in/index.php/ijcrr/article/download/339/170 Fahim MA, Saleh A. Residents as teachers: The concept, rationale and challenges. Ibnosina J Med Biomed Sci [Internet]. 2014 Apr [cited 2025 Jan 17];06(02):62–6. Available from: http://www.thieme-connect.de/DOI/DOI?10.4103/1947-489X.210365 Alsalamah A, Callinan C. The Kirkpatrick model for training evaluation: bibliometric analysis after 60 years (1959–2020). Ind Commer Train [Internet]. 2021 Aug 23 [cited 2025 Jan 17];54(1):36–63. Available from: https://www.emerald.com/insight/content/doi/10.1108/ict-12-2020-0115/full/html NBME Item-Writing Guide | NBME [Internet]. [cited 2025 Jan 20]. Available from: https://www.nbme.org/educators/item-writing-guide Hawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB, et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ [Internet]. 2015 Nov [cited 2024 Oct 29];49(11):1086–102. Available from: https://onlinelibrary.wiley.com/doi/10.1111/medu.12831 Kasthuri A. Challenges to Healthcare in India - The Five A’s. Indian J Community Med Off Publ Indian Assoc Prev Soc Med [Internet]. 2018 Sep [cited 2024 Oct 28];43(3):141. Available from: https://pmc.ncbi.nlm.nih.gov/articles/PMC6166510/ Mahajan R, Saiyad S. Postgraduate Medical Education Regulations 2023: A Critical Review. Int J Appl Basic Med Res [Internet]. 2024 [cited 2024 Dec 28];14(1):1–6. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10947754/ MD_Pathology_(revised)-2. Harrell H, Wipf J, Aronowitz P, Rencic J, Smith DG, Hingle S, et al. Resident as Teacher Curriculum. MedEdPORTAL [Internet]. [cited 2025 Jan 17];11:10001. Available from: https://www.mededportal.org/doi/10.15766/mep_2374-8265.10001 Grainger R, Dai W, Osborne E, Kenwright D. Medical students create multiple-choice questions for learning in pathology education: a pilot study. BMC Med Educ [Internet]. 2018 Dec [cited 2025 Jan 17];18(1):201. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-018-1312-1 Herrero JI, Lucena F, Quiroga J. Randomized study showing the benefit of medical study writing multiple choice questions on their learning. BMC Med Educ [Internet]. 2019 Dec [cited 2025 Jan 17];19(1):42. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-019-1469-2 Sibomana I, Karenzi ID, Niyongombwa I, Byiringiro JC, Gashegu J, Ntirenganya F. Use of Student-Generated Multiple Choice Questions to Enhance Team-Based Learning of Anatomy at the University of Rwanda. Adv Med Educ Pract. 2020;11:825–32. Agarwal A, Khattak O, Esmaeel SE, Fahmy E, Bayomy NA, Mehmood SI, et al. Efficacy of Faculty Development Training Workshops (FDTWs) on Writing High-Quality Multiple-Choice Questions at Northern Border University (NBU) in the Kingdom of Saudi Arabia (KSA). Cureus [Internet]. [cited 2025 Jan 17];16(6):e62607. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11257642/ Rowat J, Johnson K, Antes L, White K, Rosenbaum M, Suneja M. Successful implementation of a longitudinal skill-based teaching curriculum for residents. BMC Med Educ [Internet]. 2021 Jun 15 [cited 2025 Jan 17];21(1):346. Available from: https://doi.org/10.1186/s12909-021-02765-x Przymuszała P, Piotrowska K, Lipski D, Marciniak R, Cerbin-Koczorowska M. Guidelines on Writing Multiple Choice Questions: A Well-Received and Effective Faculty Development Intervention. Sage Open [Internet]. 2020 Jul 1 [cited 2025 Jan 17];10(3):2158244020947432. Available from: https://doi.org/10.1177/2158244020947432 Busari JO, Prince KJAH, Scherpbier AJJA, Van Der Vleuten CPM, Essed GGM. How residents perceive their teaching role in the clinical setting: a qualitative study. Med Teach. 2002 Jan;24(1):57–61. Gottlieb M, Bailitz J, Fix M, Shappell E, Wagner MJ. Educator’s blueprint: A how‐to guide for developing high‐quality multiple‐choice questions. AEM Educ Train [Internet]. 2023 Jan 24 [cited 2025 Jan 18];7(1):e10836. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9873868/ Janzen BW, Sommerfeld C, Gooi ACC. Effectiveness of e-Learning in a Medical School 2.0 Model: Comparison of Item Analysis for Student-Generated vs. Faculty-Generated Multiple-Choice Questions. Stud Health Technol Inform. 2019;257:184–8. Pham H, Court-Kowalski S, Chan H, Devitt P. Writing Multiple Choice Questions-Has the Student Become the Master? Teach Learn Med. 2023;35(3):356–67. Walsh JL, Harris BHL, Denny P, Smith P. Formative student-authored question bank: perceptions, question quality and association with summative performance. Postgrad Med J. 2018 Feb;94(1108):97–103. Harris BHL, Walsh JL, Tayyaba S, Harris DA, Wilson DJ, Smith PE. A novel student-led approach to multiple-choice question generation and online database creation, with targeted clinician input. Teach Learn Med. 2015;27(2):182–8. Gooi ACC, Sommerfeld CS. Medical school 2.0: How we developed a student-generated question bank using small group learning. Med Teach. 2015;37(10):892–6. Lahti J, Salamon M, Farhat J, Varkey T. Multiple choice question writing and medical students: a systematic literature review [version 1; peer review: awaiting peer review]. MedEdPublish. 2023 Jul 17;13. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-5867554","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":407991686,"identity":"5408fe74-2e44-484f-86a2-6a9e78f757c0","order_by":0,"name":"Reshma G. Kini","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA5UlEQVRIie3PvwqCQBzA8Tt+cC7qnDj0CoZTUPQqhUNLvYMgnEs1K0G9hTQmrlc9wG/yDXTLaOh0a8h0C7ov3D+5D3iEqFS/2FkO6hMyALkv5IFpfUga1QQ6E7mCXn/5Rky85Hl1SoZWaKTZ5JEMTSC0KFefiXVduq4hcLQHc56tdzjiQMCKk8/EEYzZlCM9gO5k6w1SSRgY7USrKo6zhow3OOtCGDE4LvY1IXdcfCWWYGBL4sWB7qRbHz0ONGh9iykYLeWPTaObcIv7E6fHMEiLsoW8R3kz+13v1z37XFapVKp/6QWoHFF+VMxaRwAAAABJRU5ErkJggg==","orcid":"","institution":"Father Muller Medical College","correspondingAuthor":true,"prefix":"","firstName":"Reshma","middleName":"G.","lastName":"Kini","suffix":""},{"id":407991687,"identity":"3961fc47-ffd7-43db-9f02-c9cd7ef1dbf9","order_by":1,"name":"Nisha J Marla","email":"","orcid":"","institution":"Father Muller Medical College","correspondingAuthor":false,"prefix":"","firstName":"Nisha","middleName":"J","lastName":"Marla","suffix":""},{"id":407991688,"identity":"2ead7950-941f-4034-a615-c806ca26ac52","order_by":2,"name":"Christol Blanch Moras","email":"","orcid":"","institution":"Father Muller Medical College","correspondingAuthor":false,"prefix":"","firstName":"Christol","middleName":"Blanch","lastName":"Moras","suffix":""}],"badges":[],"createdAt":"2025-01-20 16:23:25","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-5867554/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-5867554/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":75000701,"identity":"e925e96b-10e4-4256-85af-09a093f9c7fa","added_by":"auto","created_at":"2025-01-29 09:43:47","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":219831,"visible":true,"origin":"","legend":"\u003cp\u003eTitle: Frequency distribution of responses of Kirkpatrick Level 1 on Likert scale.\u003c/p\u003e\n\u003cp\u003eLegend: Response include content appropriateness, session objectives, relevance, session timing, and overall enjoyment\u003c/p\u003e","description":"","filename":"Figure1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-5867554/v1/9fc05e60da8807ab78d872e0.jpg"},{"id":81179956,"identity":"1ce5b8dc-b53b-405d-a95a-820b647b836c","added_by":"auto","created_at":"2025-04-23 07:09:51","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":997598,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-5867554/v1/49079c1a-93eb-4427-bcd3-fe7002edf51b.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Overcoming Challenges in Implementation of Competency- Based Medical Education in India: Leveraging Postgraduates for Multiple Choice Item Design for Undergraduate Assessments in Pathology","fulltext":[{"header":"Background","content":"\u003cp\u003eCompetency-Based Medical Education (CBME) represents a paradigm shift in medical training, emphasizing the achievement of specific competencies over time-based progression. In India, the National Medical Commission (NMC) has mandated the implementation of CBME in undergraduate medical education from 2019, introducing new challenges and demands for faculty development, curriculum design, and assessment standardization. With over 700 medical colleges in India, implementing CBME on such a large scale presents significant logistical and resource challenges. These include a shortage of trained faculty and a lack of standardization in assessment practices, particularly in regions with limited institutional resources.[1\u0026ndash;3]\u003c/p\u003e \u003cp\u003eTo address these challenges, leveraging pathology postgraduates as educators represents an innovative and practical solution. Postgraduates, often tasked with supporting undergraduate teaching, are well-positioned to contribute meaningfully to the implementation of CBME if provided with appropriate training. Developing their skills in assessment design can not only enhance their own teaching competencies but also alleviate some of the faculty shortages that hinder CBME implementation. Specifically, training postgraduates in the construction and validation of Multiple-Choice Items (MCIs) can improve the quality of undergraduate assessments, ensuring alignment with curricular objectives and competency-based frameworks.[4,5]\u003c/p\u003e \u003cp\u003eWe use the framework of \u0026lsquo;Residents as Teachers\u0026rsquo;. In western countries structured programs have been published for training residents in teaching methodologies. In absence of such data from India we conducted a preliminary survey across 11 medical colleges running postgraduate pathology program(unpublished). The survey revealed while all participating institutions reported involving PGs in undergraduate teaching, only 35% provided structured teaching-methodology training and none involved them in undergraduate assessment.[5\u0026ndash;7]\u003c/p\u003e \u003cp\u003eThis study aimed to evaluate the effectiveness of a structured training program designed to improve assessment literacy by enhancing pathology postgraduates' competencies in developing MCIs for undergraduate assessments. Using Kirkpatrick Levels 1 and 2 as the evaluation framework, the program assessed participants' reaction and learning outcomes. Additionally, an unconventional component\u0026mdash;product evaluation\u0026mdash;was incorporated to analyze the quality of the MCQs produced during the program. The broader goal was to explore how such training programs can address CBME implementation challenges by leveraging postgraduate educators and improving the standardization of assessment practices.[8,9]\u003c/p\u003e \u003cp\u003eIn this manuscript, we present the outcomes of the program, including participants' improvements in knowledge, confidence, and MCI development skills, alongside insights from product evaluation. The findings contribute to the growing discourse on innovative strategies for CBME implementation, particularly in CBME na\u0026iuml;ve regions experiencing resource challenges.[10,11]\u003c/p\u003e"},{"header":"Methodology","content":"\u003cp\u003e\u003cem\u003e\u003cu\u003eStudy Design and Setting\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThis was a prospective, interventional study conducted at a single institution in India. The study aimed to evaluate the effectiveness of a structured training program designed to enhance the competencies of first-year pathology postgraduates in developing Multiple Choice Items (MCIs) for undergraduate medical education. The program was conducted over a span of 3 weeks.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cu\u003eParticipants\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe study included nine first-year postgraduate students enrolled in the Department of Pathology. All participants provided informed consent prior to their inclusion in the program. Participants were selected based on their enrollment in the department and their involvement in teaching activities for undergraduate students.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cu\u003eProgram Structure\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe training program consisted of nine interactive sessions, each lasting for 1-1.5-hour duration The design of the sessions and outcomes is provided in Table 1\u003c/p\u003e\n\u003cp\u003eTable 1: The postgraduate training program with sequence and structure of individual sessions of individual\u0026nbsp;\u003c/p\u003e\n\u003ctable border=\"1\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eSession Number\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTitle/Topic\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eLearning Objectives\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eKey Content/Activities\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMode of Delivery\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eDuration\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eResources/Tools Used\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eAssessment Component\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eOutput/Deliverable\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eKey Outcomes\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eIntroduction to CBME \u003csup\u003ea\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Understand the principles of CBME and its significance.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eOverview of CBME, its goals, and alignment with assessments.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eInteractive Lecture\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePresentation Slides\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePre-session quiz\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eUnderstanding of CBME framework.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eParticipants oriented to CBME principles.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e2\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eBloom\u0026apos;s Taxonomy and MCIs\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Apply Bloom\u0026rsquo;s taxonomy to design questions assessing LOTS\u003csup\u003ec\u0026nbsp;\u003c/sup\u003eand HOTS\u003csup\u003ed\u003c/sup\u003e.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eExplanation of cognitive levels and examples of MCIs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eWorkshop\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eBloom\u0026rsquo;s taxonomy chart, examples\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eGroup activity on taxonomy alignment.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eDraft MCIs classified by Bloom\u0026rsquo;s levels.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eImproved ability to classify cognitive skills.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e3\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eMCI Writing Principles\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Learn guidelines for writing effective MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eBest practices for item writing and avoiding common flaws.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eHands-on Session\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eItem-writing guidelines document\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePeer-reviewed MCI drafts\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eDraft MCIs adhering to guidelines.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eEnhanced quality of drafted MCQIs.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e4\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eIdentifying Technical Flaws\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Identify and eliminate technical flaws in MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eExamples of flawed questions and solutions.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eGroup Discussion\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eExample question bank\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePeer review session\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eRevised MCQs with fewer technical flaws.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eParticipants learned to recognize flaws.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e5\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eValidation of MCQs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Ensure content, face, and construct validity in MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eSteps in validating MCQs; expert feedback.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eInteractive Lecture\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eValidation checklist\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eSME validation feedback\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eValidated MCQs ready for use.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eParticipants gained validation skills.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e6\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eDifficulty and Discrimination\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Analyze MCQs using difficulty and discrimination indices.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eExplanation of indices, examples, and calculations.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eHands-on Workshop\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eSpreadsheet for index calculations\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePost-session group analysis\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eMCQs with analyzed indices.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eUnderstanding of quality metrics for MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e7\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eBlueprinting for Assessments\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Develop assessment blueprints aligned with curricular objectives.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eSteps to create a blueprint and align with competencies.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eWorkshop\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eBlueprint templates, curriculum docs\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eBlueprint review\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eBlueprinted assessments.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eAssessments aligned with objectives.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e8\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003ePeer and Expert Review\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Incorporate peer and expert feedback to improve MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eStructured review process; feedback integration.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eCollaborative Session\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePeer review rubric\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eFeedback forms\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eRevised MCQs incorporating feedback.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eEnhanced quality through iterative improvement.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 45px;\"\u003e\n \u003cp\u003e9\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 123px;\"\u003e\n \u003cp\u003eTechnology for Assessment\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e- Use digital tools for creating, deploying, and analyzing MCQs.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eDemonstration of Google Forms and other tools.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eInteractive Demo\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e1.5 hours\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eGoogle Forms, analysis software\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003ePost-session quiz\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eDigitized MCQs for assessments.\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 83px;\"\u003e\n \u003cp\u003eParticipants equipped with tech skills.\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003ea-Competency Based Medical Education b- Multiple Choice Items c- Lower Order Thinking Skills d- Higher Order Thinking Skills\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cu\u003eEvaluation Framework\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe effectiveness of the training program was evaluated using Kirkpatrick\u0026rsquo;s model:\u003c/p\u003e\n\u003cp\u003eLevel 1: Reaction\u003c/p\u003e\n\u003cp\u003eParticipant satisfaction with the program was assessed through structured feedback forms administered at the end of the program. Feedback included Likert-scale ratings on session relevance, content, objectives, and duration, as well as open-ended questions to gather qualitative insights.\u003c/p\u003e\n\u003cp\u003eLevel 2: Learning- Surveys\u003c/p\u003e\n\u003cp\u003eParticipants\u0026rsquo; improvements in knowledge and confidence were evaluated using a pre- and post-program survey. The survey consisted of 20 items addressing key competencies, such as understanding Bloom\u0026rsquo;s taxonomy, identifying technical flaws, and applying validation. A retro-prequestionnaire addressing their knowledge and confidence in the major aspects of training component was also taken.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLevel 2: Learning- Product Evaluation\u003c/p\u003e\n\u003cp\u003eThe MCIs created by participants during the program were subjected to peer validation. Following this the MCIs were evaluated by subject matter experts (SMEs). MCQs were assessed for validity. A set of 30 randomly selected MCQs which needed no validation were deployed as a test for undergraduate students. Difficulty indices, and discrimination indices was calculated from the responses.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cu\u003eStatistical Analysis\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eQuantitative data from the pre- and post-program surveys were analyzed using Wilcoxon Signed-Rank Test] to determine the significance of changes in knowledge and confidence. Descriptive statistics were used to summarize feedback responses. The quality of MCIs was analyzed based on the percentage of items meeting predefined thresholds for difficulty (0.31\u0026ndash;0.70) and discrimination (\u0026ge;0.30). The statistical analysis was done using SPSS software version 21.\u003c/p\u003e\n\u003cp\u003eThe open-ended feedback was analyzed qualitatively through thematic analysis. Responses collected via structured feedback forms were reviewed to identify recurring themes, highlighting positive aspects such as training usefulness, session clarity, and relevance to assessment. Suggestions for improvement, like session scheduling and content depth, were categorized. Key themes and illustrative quotes were summarized and incorporated into the results to provide qualitative insights into participants\u0026apos; reactions.\u003c/p\u003e\n\u003cp\u003e\u003cem\u003e\u003cu\u003eEthical Considerations\u003c/u\u003e\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003eThe study was approved by the institutional ethics committee. Participants were informed about the purpose of the study and provided written consent. Confidentiality of participant responses and MCIs was maintained throughout the study.\u003c/p\u003e"},{"header":"Results","content":"\u003ch4\u003eParticipant Demographics\u003c/h4\u003e\n\u003cp\u003eNine first-year pathology postgraduate students participated in the study, attending all nine sessions of the training program. Their participation rate was 100%, ensuring consistency in feedback and evaluation.\u003c/p\u003e\n\u003ch3\u003e\u003cstrong\u003e\u003cem\u003e\u003cu\u003eKirkpatrick Level 1: Reaction\u003c/u\u003e\u003c/em\u003e\u003c/strong\u003e\u003c/h3\u003e\n\u003cp\u003eParticipants\u0026apos; reactions were captured using a 5-point Likert scale questionnaire and open-ended feedback. Responses evaluated content appropriateness, session objectives, relevance, session timing, and overall enjoyment. (Fig 1)\u003c/p\u003e\n\u003cp\u003eOpen-ended responses to the question \u0026quot;What was good about the sessions?\u0026quot; were thematically analyzed. Six key themes emerged:\u003c/p\u003e\n\u003col start=\"1\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eAwareness of Complexity in MCQ Construction\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eParticipants appreciated learning about the multiple factors involved in framing MCQs.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;I realized that there is so much behind preparing MCQs only after attending these sessions.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eKnowledge Enhancement and New Insights\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eThe sessions introduced new concepts and expanded participants\u0026apos; understanding.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Learnt new concepts which were fascinating.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003ePractical and Relevant Examples\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eThe use of real-world examples was highlighted as particularly beneficial.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Relevant examples on how to improve MCQ formation.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eClarity and Structured Delivery\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eParticipants praised the clarity and structured approach of the sessions.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Explained in detail from the scratch.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eApplicability to Broader Contexts\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eThe knowledge gained was seen as relevant beyond the immediate training.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;It broadened my knowledge on how parallel entrance coaching works.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eInformative and Engaging Content\u003c/strong\u003e:\u003cul type=\"circle\"\u003e\n \u003cli\u003eParticipants described the sessions as informative and engaging.\u003c/li\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;The sessions were informative, and a different aspect of learning and question preparation was learnt.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThis thematic analysis demonstrates the effectiveness of the training in meeting its objectives while providing practical, engaging, and insightful learning experiences.\u003c/p\u003e\n\u003cp\u003eParticipant feedback on potential improvements highlighted the following themes:\u003c/p\u003e\n\u003col start=\"1\" type=\"1\"\u003e\n \u003cli\u003e\u003cstrong\u003eSession Scheduling\u003c/strong\u003e: Continuous, back-to-back sessions were found to be exhausting, with suggestions for more spaced-out sessions and fixed goals.\u003cul type=\"circle\"\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Preferably spaced-out sessions. And fixed goals for each class would have been ideal.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eTime Management\u003c/strong\u003e: Improved allocation of time for individual sessions was recommended.\u003cul type=\"circle\"\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Time management.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eDepth of Content\u003c/strong\u003e: Participants expressed interest in more detailed discussions on question-framing methods.\u003cul type=\"circle\"\u003e\n \u003cli\u003e\u003cem\u003e\u0026quot;Few more sessions on question-framing methods might be of use.\u0026quot;\u003c/em\u003e\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThis feedback reflects a desire for optimized scheduling and deeper exploration of advanced topics to enhance the learning experience.\u003c/p\u003e\n\u003ch4\u003e\u003cstrong\u003eSelf-Perceived Changes\u003c/strong\u003e\u003c/h4\u003e\n\u003cp\u003eParticipants\u0026apos; perceived changes in knowledge and confidence were evaluated using a retro-prequestionnaire. The results are presented in Table 2. The results demonstrate that the training program had a meaningful and positive impact on the participants\u0026rsquo; understanding of key educational concepts and their confidence in implementing these skills. The statistical significance supports the effectiveness of the intervention.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 2. Self-Perceived Knowledge and Confidence\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\" width=\"672\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003eKnowledge on Learning Domains before vs after session\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 88px;\"\u003e\n \u003cp\u003eKnowledge regarding Blooms taxonomy before vs after the sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eKnowledge HOTS\u003csup\u003ec\u003c/sup\u003e before vs after the sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003eConfidence in making MCQs for HOTS and LOTS\u003csup\u003ed\u0026nbsp;\u003c/sup\u003ebefore vs after the sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003eKnowledge on concept of validity before vs after the sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 132px;\"\u003e\n \u003cp\u003eConfidence to perform post-validation before vs after the sessions\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eZ\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e-2.714\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 88px;\"\u003e\n \u003cp\u003e-2.598\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e-2.555\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003e-2.565\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003e-2.565\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 132px;\"\u003e\n \u003cp\u003e-2.558\u003csup\u003eb\u003c/sup\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003eAsymp. Sig. (2-tailed)\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 104px;\"\u003e\n \u003cp\u003e.007\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 88px;\"\u003e\n \u003cp\u003e.009\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 84px;\"\u003e\n \u003cp\u003e.011\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003e.010\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 90px;\"\u003e\n \u003cp\u003e.010\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd valign=\"top\" style=\"width: 132px;\"\u003e\n \u003cp\u003e.011\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd colspan=\"7\" valign=\"top\" style=\"width: 672px;\"\u003e\n \u003cp\u003e\u003csup\u003ea.\u003c/sup\u003e Wilcoxon Signed Ranks Test \u003csup\u003eb.\u003c/sup\u003e Based on negative ranks. \u003csup\u003ec\u003c/sup\u003e. HOTS- Higher Order Thinking Skills \u003csup\u003ed\u0026nbsp;\u003c/sup\u003eLOTS- Lower Order Thinking Skills Significance at p \u0026lt;0.05\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eParticipants created 340 MCIs, in topics commonly taught to undergraduates. The which were peer-reviewed and also assessed by subject matter experts (SMEs).\u003c/p\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eRevalidation\u003c/strong\u003e:\u003c/li\u003e\n\u003c/ul\u003e\n\u003cul type=\"disc\"\u003e\n \u003cul type=\"circle\"\u003e\n \u003cli\u003e\u003cstrong\u003e43.2%\u003c/strong\u003e of MCIs required SME revalidation after peer review.\u003c/li\u003e\n \u003cli\u003eReasons included issues with:\u003cul type=\"square\"\u003e\n \u003cli\u003eFace validity (26%),\u003c/li\u003e\n \u003cli\u003eContent validity (15%),\u003c/li\u003e\n \u003cli\u003eConstruct validity (69%).\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/li\u003e\n \u003c/ul\u003e\n\u003c/ul\u003e\n\u003cul\u003e\n \u003cli\u003e\u003cstrong\u003eItem Metrics\u003c/strong\u003e:\u003c/li\u003e\n\u003c/ul\u003e\n\u003cul type=\"disc\"\u003e\n \u003cul type=\"circle\"\u003e\n \u003cli\u003e\u003cstrong\u003eDifficulty Index\u003c/strong\u003e: Acceptable in (213) 62.5%. Easy in 122 (35.8) %\u003c/li\u003e\n \u003cli\u003e\u003cstrong\u003eDiscrimination Index\u003c/strong\u003e: Good/Excellent in 75%.\u003c/li\u003e\n \u003c/ul\u003e\n\u003c/ul\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study evaluated a structured training program designed to enhance pathology postgraduates\u0026apos; abilities in developing Multiple Choice Items (MCIs) for undergraduate assessments, utilizing Kirkpatrick\u0026apos;s Levels 1 and 2 model of training evaluation. The findings indicate significant improvements in participants\u0026apos; knowledge, confidence, and skills related to MCI construction, aligning with educational theories and corroborating existing literature.[8]\u003c/p\u003e\n\u003ch4\u003eLeveraging Postgraduates for CBME Implementation\u003c/h4\u003e\n\u003cp\u003eThe findings underscore the feasibility of leveraging postgraduates as educators to address challenges in implementing CBME. As of January 2025, India has approximately 779 medical colleges. Implementing Competency-Based Medical Education (CBME) across such a vast number of institutions presents significant challenges, including faculty development, infrastructure enhancement, and standardization of assessment methods. In this context, training postgraduate students in assessment methodologies could be invaluable. However, it is essential to consider their readiness and ensure they receive adequate support to perform effectively in these roles..[4\u0026ndash;6]\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAlso, the program addresses a critical gap identified in prior needs assessment surveys, where many institutions employed postgraduates in teaching roles without formal training. This initiative fulfills an institutional obligation to provide postgraduates with the competencies needed to design robust, valid assessment tools The ability to design and validate Multiple Choice Items (MCIs) is a valuable skill, particularly with the increasing reliance on standardized assessments like the National Exit Test. Furthermore, introducing PGs to robust assessment practices early in their careers promotes lifelong skills in education and evaluation, and enhance their learning of pathology. [7,12\u0026ndash;17]\u003c/p\u003e\n\u003cp\u003eSeveral studies have concluded that training faculty in developing high quality MCQs is effective. The uniqueness of our program is that we aimed at enhancement of assessment literacy among first year postgraduates. Our program included components of blueprinting, using digital tools such as google forms for assessment and fundamental concepts of Bloom\u0026rsquo;s taxonomy, Competency- based education, deriving Specific learning objective. And MCQ validation. This was aimed at holistic development of a postgraduate as an assessor of undergraduates. However the focus of our evaluation was on the quality of the MCIs produced.[18\u0026ndash;20]\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe feasibility of conducting this program was because the sessions could be conducted at the department by the department faculty, and it was of short duration. This is an important consideration when integrating the program in the current postgraduate curriculum.\u003c/p\u003e\n\u003ch4\u003eInsights from Participant Reactions and Learning Outcomes\u003c/h4\u003e\n\u003cp\u003eThe effectiveness of the program is evidenced in multiple ways. The improvements observed in participants\u0026apos; knowledge and confidence in creating MCIs are consistent with foundational educational theories such as Kolb\u0026rsquo;s experiential learning Theory emphasizes learning through a cycle of experience, reflection, conceptualization, and experimentation. The program\u0026rsquo;s emphasis on practical tasks, such as creating and validating MCIs, enabled participants to apply theoretical principles in real-world scenarios while refining their skills based on peer and expert feedback. The secondary benefit of deeper understanding of the subject and enhancing self-learning could be deduced by the quality of the MCIs produced.[18,21]\u003c/p\u003e\n\u003cp\u003eWe focused the study on evaluating the collective impact of the training program on the participants as a group, hence we did not analyse individual postgraduate\u0026rsquo;s performances Given the relatively homogenous background of the participants\u0026mdash;all being first-year pathology postgraduates\u0026mdash;individual-level analysis was not expected to yield additional insights beyond those provided by group-level trends. Moreover, with a small cohort size, categorizing individual performances might have limited statistical utility and could detract from the broader implications of the findings.\u003c/p\u003e\n\u003ch4\u003eProduct Evaluation and Quality of MCIs\u003c/h4\u003e\n\u003cp\u003eInsights from the product evaluation highlight that a significant percentage of MCQs required revalidation by subject matter experts (SMEs). Furthermore, the majority of the MCQs produced were focused on assessing lower order thinking skills, predominantly at the \u0026quot;understanding\u0026quot; level of Bloom\u0026apos;s taxonomy. Several factors could account for these findings. First, all participants were first-year postgraduates, and their limited in-depth knowledge of pathology may have constrained their ability to generate high-quality MCQs that assess higher-order thinking skills. Second, the absence of a longitudinal program with ongoing mentorship and support may have hindered their ability to refine and improve their MCQs over time. Finally, the SMEs involved in the validation process were highly experienced, potentially creating a substantial gap in expertise between the two groups. This disparity could have contributed to the higher revalidation rates observed during the evaluation.[22]\u003c/p\u003e\n\u003cp\u003eIn contrast to this majority of the MCQs had acceptable difficulty and discriminatory Indices making our findings comparable to those of Pham et al and Janzen et alwho compared the student and faculty generated MCIs.[23,24]\u003c/p\u003e\n\u003cp\u003eThis could be a promising trend as leveraging students in generating MCQs could overcome the challenge of generating question banks for continuous assessments mandated by the NMC[23,25\u0026ndash;27].\u003c/p\u003e\n\u003cp\u003eHowever, certain limitations must be acknowledged. The small sample size and single-institution setting may limit the generalizability of the findings. Future research should include larger, more diverse cohorts across multiple institutions to strengthen external validity. Additionally, while this study focused on immediate improvements in knowledge and confidence, it did not evaluate the long-term retention of skills or their application in actual educational settings. Longitudinal studies are recommended to assess the sustained impact of such training programs. Finally, the study was specific to pathology postgraduates, and exploring the effectiveness of similar programs in other medical specialties could provide valuable insights into the broader applicability of the training model[28].\u003c/p\u003e\n\u003ch4\u003e\u0026nbsp;Implications for Medical Education\u003c/h4\u003e\n\u003cp\u003eThis study demonstrates the potential of structured training programs to enhance postgraduate competencies in assessment design, contributing to the broader goals of CBME. The findings highlight the importance of incorporating product evaluation as an additional dimension in program evaluation, bridging the gap between learning outcomes and real-world application. Furthermore, the scalability of such programs across institutions and specialties could address systemic challenges in medical education, particularly in resource-constrained settings.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eIn conclusion, this study highlights the feasibility and effectiveness of a structured training program in enhancing postgraduates' skills in MCI development. By aligning with the NMC\u0026rsquo;s mandate and integrating evidence-based educational practices, the program demonstrates its potential to improve postgraduate teaching competencies and undergraduate assessment quality, contributing to the broader goals of competency-based medical education.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cul\u003e\n \u003cli\u003eEthics approval was obtained institutional ethics committee and consent to participate was obtained from the participants\u003c/li\u003e\n \u003cli\u003eConsent for publication- not applicable\u003c/li\u003e\n \u003cli\u003eAvailability of data and materials- The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.\u0026nbsp;\u003c/li\u003e\n \u003cli\u003eCompeting interests- authors declare no competing intrests\u003c/li\u003e\n \u003cli\u003eFunding- the study was self-funded\u003c/li\u003e\n \u003cli\u003eAuthors\u0026apos; contributions- All the authors contributed to the article as per the authorship criteria\u003c/li\u003e\n \u003cli\u003eAcknowledgements: nil\u003c/li\u003e\n \u003cli\u003eClinical trial number: not applicable\u003c/li\u003e\n\u003c/ul\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eNMC, National Exit Test Regulations, 2023 | NMC [Internet]. [cited 2025 Jan 17]. Available from: https://www.nmc.org.in/rules-regulations/nmc-national-exit-test-regulations-2023/\u003c/li\u003e\n\u003cli\u003ePG Curricula | NMC [Internet]. [cited 2025 Jan 17]. Available from: https://www.nmc.org.in/information-desk/for-colleges/pg-curricula-2/\u003c/li\u003e\n\u003cli\u003eUG Curriculum | NMC [Internet]. [cited 2024 Nov 17]. Available from: https://www.nmc.org.in/information-desk/for-colleges/ug-curriculum/\u003c/li\u003e\n\u003cli\u003eKurien M, Thomas K. Competency Based Medical Education (CBME) in India: Clinicians\u0026rsquo; Multifaceted Challenges. Indian J Otolaryngol Head Neck Surg [Internet]. 2024 Aug 1 [cited 2025 Jan 18];76(4):3730\u0026ndash;4. Available from: https://doi.org/10.1007/s12070-024-04675-y\u003c/li\u003e\n\u003cli\u003eMcKinney CM, Hart R, Patterson AC, McKinney CM, Hart R, Patterson AC. Beyond Residents-as-Teachers: The Development of an Advanced Medical Education Pilot Elective for Pediatric Residents. Cureus [Internet]. 2023 Jun 25 [cited 2025 Jan 17];15(6). Available from: https://www.cureus.com/articles/161105-beyond-residents-as-teachers-the-development-of-an-advanced-medical-education-pilot-elective-for-pediatric-residents\u003c/li\u003e\n\u003cli\u003eIsmail MA, Hamed OAE, Ismail SA. Residents as Professionalism Teachers and Assessors; A Pilot Study on Implementing a Faculty Development Program Leading to Curriculum Development. Int J Contemp Res Rev [Internet]. 2017 Oct 15 [cited 2025 Jan 18];8(10). Available from: http://ijcrr.in/index.php/ijcrr/article/download/339/170\u003c/li\u003e\n\u003cli\u003eFahim MA, Saleh A. Residents as teachers: The concept, rationale and challenges. Ibnosina J Med Biomed Sci [Internet]. 2014 Apr [cited 2025 Jan 17];06(02):62\u0026ndash;6. Available from: http://www.thieme-connect.de/DOI/DOI?10.4103/1947-489X.210365\u003c/li\u003e\n\u003cli\u003eAlsalamah A, Callinan C. The Kirkpatrick model for training evaluation: bibliometric analysis after 60 years (1959\u0026ndash;2020). Ind Commer Train [Internet]. 2021 Aug 23 [cited 2025 Jan 17];54(1):36\u0026ndash;63. Available from: https://www.emerald.com/insight/content/doi/10.1108/ict-12-2020-0115/full/html\u003c/li\u003e\n\u003cli\u003eNBME Item-Writing Guide | NBME [Internet]. [cited 2025 Jan 20]. Available from: https://www.nbme.org/educators/item-writing-guide\u003c/li\u003e\n\u003cli\u003eHawkins RE, Welcher CM, Holmboe ES, Kirk LM, Norcini JJ, Simons KB, et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ [Internet]. 2015 Nov [cited 2024 Oct 29];49(11):1086\u0026ndash;102. Available from: https://onlinelibrary.wiley.com/doi/10.1111/medu.12831\u003c/li\u003e\n\u003cli\u003eKasthuri A. Challenges to Healthcare in India - The Five A\u0026rsquo;s. Indian J Community Med Off Publ Indian Assoc Prev Soc Med [Internet]. 2018 Sep [cited 2024 Oct 28];43(3):141. Available from: https://pmc.ncbi.nlm.nih.gov/articles/PMC6166510/\u003c/li\u003e\n\u003cli\u003eMahajan R, Saiyad S. Postgraduate Medical Education Regulations 2023: A Critical Review. Int J Appl Basic Med Res [Internet]. 2024 [cited 2024 Dec 28];14(1):1\u0026ndash;6. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10947754/\u003c/li\u003e\n\u003cli\u003eMD_Pathology_(revised)-2. \u003c/li\u003e\n\u003cli\u003eHarrell H, Wipf J, Aronowitz P, Rencic J, Smith DG, Hingle S, et al. Resident as Teacher Curriculum. MedEdPORTAL [Internet]. [cited 2025 Jan 17];11:10001. Available from: https://www.mededportal.org/doi/10.15766/mep_2374-8265.10001\u003c/li\u003e\n\u003cli\u003eGrainger R, Dai W, Osborne E, Kenwright D. Medical students create multiple-choice questions for learning in pathology education: a pilot study. BMC Med Educ [Internet]. 2018 Dec [cited 2025 Jan 17];18(1):201. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-018-1312-1\u003c/li\u003e\n\u003cli\u003eHerrero JI, Lucena F, Quiroga J. Randomized study showing the benefit of medical study writing multiple choice questions on their learning. BMC Med Educ [Internet]. 2019 Dec [cited 2025 Jan 17];19(1):42. Available from: https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-019-1469-2\u003c/li\u003e\n\u003cli\u003eSibomana I, Karenzi ID, Niyongombwa I, Byiringiro JC, Gashegu J, Ntirenganya F. Use of Student-Generated Multiple Choice Questions to Enhance Team-Based Learning of Anatomy at the University of Rwanda. Adv Med Educ Pract. 2020;11:825\u0026ndash;32. \u003c/li\u003e\n\u003cli\u003eAgarwal A, Khattak O, Esmaeel SE, Fahmy E, Bayomy NA, Mehmood SI, et al. Efficacy of Faculty Development Training Workshops (FDTWs) on Writing High-Quality Multiple-Choice Questions at Northern Border University (NBU) in the Kingdom of Saudi Arabia (KSA). Cureus [Internet]. [cited 2025 Jan 17];16(6):e62607. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11257642/\u003c/li\u003e\n\u003cli\u003eRowat J, Johnson K, Antes L, White K, Rosenbaum M, Suneja M. Successful implementation of a longitudinal skill-based teaching curriculum for residents. BMC Med Educ [Internet]. 2021 Jun 15 [cited 2025 Jan 17];21(1):346. Available from: https://doi.org/10.1186/s12909-021-02765-x\u003c/li\u003e\n\u003cli\u003ePrzymuszała P, Piotrowska K, Lipski D, Marciniak R, Cerbin-Koczorowska M. Guidelines on Writing Multiple Choice Questions: A Well-Received and Effective Faculty Development Intervention. Sage Open [Internet]. 2020 Jul 1 [cited 2025 Jan 17];10(3):2158244020947432. Available from: https://doi.org/10.1177/2158244020947432\u003c/li\u003e\n\u003cli\u003eBusari JO, Prince KJAH, Scherpbier AJJA, Van Der Vleuten CPM, Essed GGM. How residents perceive their teaching role in the clinical setting: a qualitative study. Med Teach. 2002 Jan;24(1):57\u0026ndash;61. \u003c/li\u003e\n\u003cli\u003eGottlieb M, Bailitz J, Fix M, Shappell E, Wagner MJ. Educator\u0026rsquo;s blueprint: A how‐to guide for developing high‐quality multiple‐choice questions. AEM Educ Train [Internet]. 2023 Jan 24 [cited 2025 Jan 18];7(1):e10836. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9873868/\u003c/li\u003e\n\u003cli\u003eJanzen BW, Sommerfeld C, Gooi ACC. Effectiveness of e-Learning in a Medical School 2.0 Model: Comparison of Item Analysis for Student-Generated vs. Faculty-Generated Multiple-Choice Questions. Stud Health Technol Inform. 2019;257:184\u0026ndash;8. \u003c/li\u003e\n\u003cli\u003ePham H, Court-Kowalski S, Chan H, Devitt P. Writing Multiple Choice Questions-Has the Student Become the Master? Teach Learn Med. 2023;35(3):356\u0026ndash;67. \u003c/li\u003e\n\u003cli\u003eWalsh JL, Harris BHL, Denny P, Smith P. Formative student-authored question bank: perceptions, question quality and association with summative performance. Postgrad Med J. 2018 Feb;94(1108):97\u0026ndash;103. \u003c/li\u003e\n\u003cli\u003eHarris BHL, Walsh JL, Tayyaba S, Harris DA, Wilson DJ, Smith PE. A novel student-led approach to multiple-choice question generation and online database creation, with targeted clinician input. Teach Learn Med. 2015;27(2):182\u0026ndash;8. \u003c/li\u003e\n\u003cli\u003eGooi ACC, Sommerfeld CS. Medical school 2.0: How we developed a student-generated question bank using small group learning. Med Teach. 2015;37(10):892\u0026ndash;6. \u003c/li\u003e\n\u003cli\u003eLahti J, Salamon M, Farhat J, Varkey T. Multiple choice question writing and medical students: a systematic literature review [version 1; peer review: awaiting peer review]. MedEdPublish. 2023 Jul 17;13. \u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Competency-Based Education, Pathology Postgraduate Education, Multiple choice items, Residents as teachers","lastPublishedDoi":"10.21203/rs.3.rs-5867554/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-5867554/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003ch2\u003eBackground\u003c/h2\u003e \u003cp\u003eThe implementation of Competency-Based Medical Education (CBME) in India has highlighted significant challenges, including faculty shortages and the need for standardized assessments. Pathology postgraduates, often involved in undergraduate teaching, represent an underutilized resource. Training them in Multiple Choice Item (MCI) development and validation can improve assessment quality while addressing gaps in CBME implementation.\u003c/p\u003e\u003ch2\u003eMethod\u003c/h2\u003e \u003cp\u003eA structured training program was conducted for nine first-year pathology postgraduates at a single institution. The program included nine interactive sessions on CBME principles, Bloom\u0026rsquo;s taxonomy, MCI construction, identification of technical flaws, validation processes, and blueprinting. Participants\u0026rsquo; reactions and self-perceived learning were evaluated using Kirkpatrick Levels 1 and 2 through pre- and post-program surveys and structured feedback forms. Product evaluation analyzed the MCIs for difficulty and discrimination indices, and the need for revalidation by subject matter experts (SMEs). Open-ended feedback was thematically analyzed.\u003c/p\u003e\u003ch2\u003eResults\u003c/h2\u003e \u003cp\u003eParticipants showed significant improvements in knowledge and confidence in MCI development (p\u0026thinsp;\u0026lt;\u0026thinsp;0.05). Product evaluation revealed that 43.2% of MCIs required revalidation, but most met acceptable thresholds for difficulty (62.5%) and discrimination (75%). Feedback highlighted the program\u0026rsquo;s structured and practical focus, while suggesting improvements in session pacing and content depth. Multiple choice items predominantly targeted lower order thinking skills, reflecting participants\u0026rsquo; novice status and the need for longitudinal support to improve question complexity.\u003c/p\u003e\u003ch2\u003eConclusion\u003c/h2\u003e \u003cp\u003eThis study demonstrates the feasibility of leveraging pathology postgraduates to address CBME implementation challenges. By equipping postgraduates with assessment design skills, the program improved undergraduate assessment quality while addressing faculty shortages. Future efforts should focus on expanding such programs across institutions, incorporating longitudinal mentorship, and evaluating their long-term impact on medical education outcomes.\u003c/p\u003e","manuscriptTitle":"Overcoming Challenges in Implementation of Competency- Based Medical Education in India: Leveraging Postgraduates for Multiple Choice Item Design for Undergraduate Assessments in Pathology","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-01-29 09:43:42","doi":"10.21203/rs.3.rs-5867554/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"487ee6d3-a3de-47e0-90dc-e26afd89511c","owner":[],"postedDate":"January 29th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2025-04-23T07:09:33+00:00","versionOfRecord":[],"versionCreatedAt":"2025-01-29 09:43:42","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-5867554","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-5867554","identity":"rs-5867554","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}