Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program

Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program Sundus Iftikhar, Syed Hasan Shoaib, Shaur Sarfaraz, Syed Kauser Ali This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4110146/v1 This work is licensed under a CC BY 4.0 License Status: Under Review Version 1 posted 10 You are reading this latest preprint version Abstract Background: MMI is a widely used method for assessing the non-cognitive skills of students, but the logistics and costs associated with organizing in-person MMI can be substantial. Virtual MMI, such as those conducted through platforms like WhatsApp Video calls, offer increased convenience, yet their reliability and quality remain uncertain. The objective of the study is to determine the reliability and quality (difficulty and discrimination indices) metrics of MMI scores conducted through WhatsApp Video call. Methods: Six MMI stations were used to assess the non-cognitive attributes of the students. In this descriptive study, the scores obtained by 678 students in MMI, F.Sc (equivalent of high school) and MDCAT were used to analyze the data. Item analysis was employed to assess quality of MMI stations. The reliability was calculated using Cronbach’s alpha and Pearson correlation (r) was performed between MDCAT scores, F.Sc. Scores and MMI scores to assess significant correlation. Results: The overall reliability of MMI in this study was 0.87 while the reliability for each interview station ranged between 0.92 - 0.95. The difficulty level of MMI stations ranged from easy (0.57) to moderately difficult (0.77). The discrimination index was found to be in the range of 0.53-0.78. Conclusion: The psychometric analysis of MMI scores demonstrated good reliability and quality (discrimination and difficulty index) with the stations showing acceptable discrimination and moderate difficulty. Hence, online multiple mini-interviews were found useful to assess non-cognitive skills for admission in MBBS program. Reliability of Results college admission test admission criteria school Figures Figure 1 Background In 2020–2021 Pakistan Medical Commission (PMC) changed the centralised admission policy of medical college admissions and allowed individual medical colleges to give 20% marks to candidates based on their admission tests or interviews ( 2 ). Shalamar Medical & Dental College chose Multiple Mini Interviews (MMI) for this purpose. In MMI the interviewee has to rotate through more than one station with the different interviewers at each ‘station’. Different tasks are posed at each ‘station’. These may involve answering questions based on scenarios, completing a practical task, or participating in a role play. The MMI allow the interviewers to assess the non-cognitive skills of the interviewee ( 3 ). This type of interview practice is being adopted widely due to high reliability of its score reported in literature ranging from α 0.7 to α 0.98( 4 )( 5 ). Since health care is now more focused on patient-centered approach therefore it is necessary that apart from knowledge, a doctor must also possess interpersonal skills, communication skills, empathy, and integrity in order to become a successful practitioner ( 6 ). Every medical college aims to select the best candidates for their MBBS programme as in a few years’ time; the healthcare of the country will depend on their competency. Nenad (2020) concluded that programs that take into account both academic and non-academic performances of the candidate are more likely to enroll students with better soft skills who otherwise would not have been shortlisted ( 7 ). Studies have reported that performance in MMI predicts future academic performance of the student in medical school ( 8 )( 9 ). Shalamar Medical and Dental College (SMDC) has been conducting MMI for admission in its Bachelor of Medicine and Bachelor of Surgery (MBBS) program from 2012–2016. The MMI was then discontinued due to the introduction of centralised admission policy by the Pakistan Medical & Dental Council (PM&DC). The reliability of the last face-to-face MMI was found to be acceptable reliability with a G coefficient of 0.70 ( 10 ). SMDC decided to conduct MMI again as the new PMC policy allowed medical colleges to give 20% marks based on their interviews. By giving 20% weightage to individual colleges, PMC placed a lot of responsibility on the shoulders of medical colleges. For this purpose, MMI are considered a better option than traditional unstructured interviews. However, in 2020 and 2021, COVID-19 pandemic made it impossible to conduct the MMI face-to-face so SMDC shifted the entire process online and conducted the MMI through WhatsApp video calls. Conducting MMI virtually was found to have several advantages like, recording of the interviews, ease in accessing the faculty and less expense to the candidates as they can take the interview from anywhere in the world ( 11 )( 12 ). In 2011, a study conducted in Australia concluded that there was no significant difference in MMI conducted remotely or face-to-face ( 13 ). However, apart from logistic and cost issues, it is important to determine the reliability of online MMI for their continued use. Therefore, the objective of the study is to determine the reliability and quality (difficulty and discrimination indices) metrics of MMI scores conducted through WhatsApp Video call. Material and Methods This descriptive study was conducted at Shalamar Medical and Dental College (SMDC). Ethical approval was granted by the Institutional Review Board (IRB) Shalamar Medical and Dental College (REF: SMDC-IRB/AL/44/2022) IRB number: 0484. Online MMI was conducted at SMDC admission for the session of 2021–2022. This interview process consisted of six stations. Each station was of 12 minutes where 3 minutes were given to the student to read the scenario. The student then answered the questions posed by the interviewer related to that scenario. The constructs to be assessed in the MMI at SMDC were selected from those that had been identified in an earlier study ( 14 ). These included “communication skills, motivation to address societal needs, ethics, empathy, critical thinking and leadership with decision making skills”. The constructs were shortlisted by the medical educationists after thorough literature search and were finalised by a committee headed by the Principal, SMDC after giving due consideration to the practicality of assessing these constructs in online format. Around 137 faculty members were trained to conduct the interviews. The training took part in two phases. In the first phase the faculty was acquainted with the philosophy and working of MMI. In the next phase the IT department trained the faculty in using WhatsApp to conduct the interviews. The students were informed about the interview process through detailed explanation on the website. A video was developed and uploaded on the College’s website to inform the students regarding the interview process. The faculty was provided with the roll numbers of the students that they had to interview with a corresponding assigned time at which they were supposed to contact the student. On the day of the interview each faculty member signed into software developed in house by the IT department of SMDC that connected the roll number of the student to WhatsApp directly on video call without the need for the faculty to dial or save each number separately. Once connected, the faculty shared the scenario with the student and started the interview. At the end of the interview the faculty entered the scores in the same software. Each interviewer scored the interviewee on 5 criteria using a 10-point scale (supplemental file A). Each criterion was pertinent to the construct being tested. Each student was scored out of a total of 240 marks, 40 marks for each station. Hence, the student rotated through 6 different stations with 6 different interviewers. A total of 731 candidates participated in MMI. The participation rate was 95% as 678 candidates appeared in all the stations of MMI. As Universal sampling was done, the sample included all the students who appeared in MMI of SMDC in the session of 2021–2022. The participants who appeared in all six stations were included in the study while the candidates who had not appeared in any one of the six stations were excluded from the study. This study was conducted from December 2022 till March 2023. The scores from each station were compiled and data obtained was then entered and analyzed descriptively using SPSS version 20. The information was kept strictly confidential. Serial numbers were used instead of names to protect the identity of the candidates. Each station’s scores mean and standard deviation was determined and compared. Overall reliability of the scores as well as reliability of each station was calculated using Cronbach’s alpha. Correlation of MMI scores with MDCAT and F.S.C scores and between stations was also analyzed. Item analysis was employed to assess quality of MMI stations. The quality measures involved calculating the difficulty (D) and discrimination (R) indices of each of the stations. Difficulty index (D) was calculated as such: The Discrimination index (R) was calculated as such: R = H/n-L/n Where: “H” is the number of top 27% candidates with a total score of 70% (28 out of 40) and above. “L” is the number of bottom 27% candidates with a total score of 70% (28 out of 40) and above. “n” is the total number of 27% of the candidates. The interpretation of difficulty and discrimination indices are given in supplemental file B. Results Descriptive Statistics was analysed with reliability (internal consistency) measures applied on all six stations by using Cronbach’s alpha (α). Pearson correlation (r) was performed between MDCAT scores, F.Sc. Scores and MMI scores to assess significant correlation. As shown in table 1 the mean scores on all items are ranged from 7.1–7.9 and SD ranged from. The reliability (Cronbach’s alpha) of scores on each construct ranged from 0.92–0.95. The overall reliability (Cronbach’s α) of scores on MMI stations was 0.87. Table 2a represents weak positive correlation between the MDCAT and the total score of the MMI exam with a significant p-value which means students who scored well on MDCAT have also scored well in MMI exam and vice versa. The F.Sc. scores correlated negatively with MMI scores. With regards to correlation between the MMI scores of each station, all stations showed positive but weak correlation with significant p-value except Communication & Counseling and Leadership which correlated negatively (Table 2b). The competency assessment covered six distinct domains, each was assessed using key metrics like the Difficulty Index and Discrimination Index to gauge the performance and effectiveness of individual items (Ethics, Leadership, Critical thinking, Ethics, Motivation, Communication skills). The easiest item were from the station of Ethics (Difficulty index = 0.77) while the most difficult item were from stations assessing Leadership (Difficulty index = 0.57). The station of Critical thinking exhibited the highest discriminating index (0.78) while the domain of Motivation displayed the lowest discriminating index (0.53). All six stations had high discrimination indices > 0.53. Figure 1 shows the comparison between Difficulty and Discrimination indices of each MMI station. Discussion The primary focus of this study was to explore the reliability and quality of Online Multiple Mini-interviews by post hoc analysis. The results suggest that online MMI had good reliability and reasonable validity. The stations were able to discriminate between high achievers and low achievers. The overall reliability of MMI scores in this study was 0.87 while the reliability for scores on individual stations was calculated to be within 0.92 and 0.95. The reliability noted in this study is in the higher range as the literature reports the reliability of MMI between 0.54 and 0.98 ( 16 ). The reliability of MMI scores conducted face-to-face at SMDC in a previous cohort was 0.7 ( 10 ). The authors believe that comparatively higher reliability reported in online MMI could be due to the familiarity of the faculty members with the process of MMI. Compared to previous cohorts, this time the faculty went through robust training sessions before MMI. Additionally, most of the faculty members had already been a part of MMI and hence were better prepared. Another change was that this time the scenarios had undergone a three-stage vetting process (via Medical educationists, faculty members, and intermediate-level students). The significant positive correlation between scores obtained at different stations (except leadership with communication skills) confirmed the convergent validity of the interviews. Although the constructs of each station are distinct, they assess the soft (non-cognitive) skills of the candidate. This explains the consistent performance of the candidates across most of the stations. Knorr et al. explain the phenomenon by emphasizing that the non-cognitive skills assessed via MMI overlap conceptually even if they are not identical ( 17 ). MDCAT and MMI scores show a weak but significant positive correlation while non-significant negative correlation was seen between scores obtained in F.Sc. and MMI. In Pakistan, there are numerous examining bodies for F.Sc. which results in the variability. Apart from different boards of examinations, Higher Education Commission of Pakistan also issues equivalence certificates to students who have taken O and A level or American high school exams. The students in this cohort had taken F.Sc. examinations from different boards, while some had submitted their equivalence certificates. However, all the candidates had taken the same MDCAT exam. Literature suggests that there is no correlation between MMI scores and academic measures like the past academic performance of a candidate (Grade point average-GPA) ( 18 ) ( 19 ). Academic performance of a candidate in the form of grade point average (GPA) or MDCAT score reflects the candidate’s performance on knowledge-based tests while MMI assesses the non-cognitive skills, hence it is expected that good performance in the knowledge-based assessment will not ensure a better performance in non-cognitive assessment. The positive correlation among the scores found in this study needs to be further explored. The difficulty level of MMI stations ranged from easy (0.57) to moderately difficult (0.77) which means that fifty-seven percent to seventy-seven percent students were able to respond satisfactorily to the questions asked at these stations.. It is quite possible that the candidates had prior knowledge of the constructs that would be assessed and hence may have undergone some sort of preparation. This indicates a need to review the scenarios and revise them or develop new ones. The discrimination index was found to be in the range of 0.53–0.78, suggesting that the stations were good discriminators. This indicates that the scenarios given in MMI were able to discriminate well between the candidates and hence contributed effectively in evaluating them on the given constructs. The limitation of this study is that the results are based on the performance of a single cohort and cannot be generalized. Secondly, video recording of the interviews was not done and only one examiner was present during the interview on each station. It is recommended that MMI should be conducted to assess the non-cognitive attributes of students. Since online MMI use fewer resources and is more feasible it may be a preferred method for conducting MMI. Conclusion The healthcare of the country depends on the competency of medical graduates so it is crucial to have robust admission criteria. The medical graduates in addition to possessing the requisite knowledge to be competent doctor must also possess the soft skills to deal with the patients and colleagues appropriately. Therefore, admission criteria for medical colleges should consist of an assessment of non-academic attributes in addition to their past assessment scores. The psychometric analysis of MMI has demonstrated adequate reliability and quality (validity, discrimination and difficulty index) and hence, MMI should continue to be used. There is a need to conduct longitudinal studies to assess the soft skills of the students throughout their medical college studies specially during clinical postings and also followed during their house job period. Declarations Ethics approval and consent to participate: Ethical approval was granted by the Institutional Review Board (IRB) Shalamar Medical and Dental College (REF: SMDC-IRB/AL/44/2022) IRB number: 0484. Consent for publication We, the authors, hereby give our consent for the publication of the manuscript titled. We affirm that all authors have contributed significantly to the study and agree to be accountable for all aspects of the work, ensuring its accuracy and integrity. Furthermore, we declare that this manuscript is not under consideration for publication elsewhere and has not been previously published in whole or in part. Availability of data and materials The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request. Additionally, any materials necessary to replicate the experiments conducted in this study will be made available upon request to the corresponding author Competing interests The authors declare that they have no competing interests Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors Authors' contributions Sundus Iftikhar: acquisition of data, interpretation of results, and manuscript writing (introduction, methodology, and discussion). Syed Hasan Shoaib: Reviewing and writing the manuscript and acquisition of data. Shaur Sarfaraz: Analysis of the data, interpretation of the work. Writing of the manuscript (results and discussion). Editing of the manuscript. Syeda Kauser: conception of the work, interpretation of the data, planning the project, editing of the manuscript. Acknowledgment: The authors gratefully acknowledge Professor Zahid Bashir for his leadership and guidance, and the IT department of Shalamar Medical and Dental College for their valuable contributions in running the system smoothly. References The 4 Types of Reliability in. Research | Definitions & Examples. Available from: https://www.scribbr.com/methodology/types-of-reliability/ . Pakistan Medical Commission. Admission Regulations (Amended) 2020–2021. Vol. 8. 2021. p. 2020–1. Medical MY. Multiple mini interview as an admission tool in higher education: insights from a systematic review. J Taibah Univ Med Sci [Internet]. 2019;14(3). https://doi.org/10.1016/j.jtumed.2019.03.006 . Dore KL, Kreuger S, Ladhani M, Rolfson D, Kurtz D, Kulasegaram K, et al. The Reliability and Acceptability of the Multiple Mini-Interview as a Selection Instrument for Postgraduate Admissions. Acad Med. 2010;85(10 SUPPL):S60–3. 10.1097/ACM.0b013e3181ed442b . Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training-A systematic review. Vol. 35, Med Teach. 2013. p. 1027–41. Available from: 10.3109/0142159X.2013.829912 http://www.tandfonline.com/doi/full/10.3109/0142159X.2013.829912 . Burgess A, Roberts C, Sureshkumar P, Mossman K, BMC Med Educ. Multiple mini interview (MMI) for general practice training selection in Australia: Interviewers’ motivation. Vol. 18,. BioMed Central Ltd.; 2018. p. 21. Available from: https://doi.org/10.1186/s12909-018-1128-z . Nenad MW. The multiple mini-interview and dental hygiene admissions: A feasible option? J Dent Educ [Internet]. 2020 Jun 1 [cited 2021 Mar 30];84(6):634–41. http://doi.wiley.com/10.1002/jdd.12114 . Jerant A, Henderson MC, Griffin E, Hall TR, Kelly CJ, Peterson EM et al. Do Admissions Multiple Mini-Interview and Traditional Interview Scores Predict Subsequent Academic Performance? A Study of Five California Medical Schools. Acad Med. 2019;94(3):388–95. DOI: 10.1097/ACM.0000000000002440 Available from: https://journals.lww.com/academicmedicine/Fulltext/2019/03000/Do_Admissions_Multiple_Mini_Interview_and.31.aspx . Kumar N, Bhardwaj S, Rahman E. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom: a cohort study. Adv Med Educ Pract. 2018;9:239. https://doi.org/10.2147/AMEP.S159412 Available from: /pmc/articles/PMC5903841/ . Haider SI, Bari MF, Ijaz S. Using multiple mini-interviews for students’ admissions in pakistan: A pilot study. Adv Med Educ Pract. 2020;11:179–85. https://doi.org/10.2147/AMEP.S246285 . Ungtrakul T, Lamlertthon W, Boonchoo B, Auewarakul C. Virtual Multiple Mini-Interview during the COVID-19 Pandemic. Med Educ. 2020;54(8):764–5. 10.1111/medu.14207 . Carroll Turpin MA, Steele K, Matuk-Villazon O, Rowland K, Dayton CB, Horn KV. Rapid transition to a virtual multiple mini-interview admissions process: A new medical school’s experience during the COVID-19 pandemic. Acad Med. 2021;96(8):1152–5. 10.1097/ACM.0000000000004179 . Tiller D, O’Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Med Educ. 2013;47(8):801–10. https://doi.org/10.1111/medu.12224 . Aziz I, Shoaib SH, Iftikhar S, Bashir MZ, Ashraf A, Shahzad M. Inducting undergraduate medical students via WhatsApp-based multiple mini interviews during COVID-19 pandemic. J Pak Med Assoc. 2022;72(11):2270–4. 10.47391/JPMA.5325 . Dela Pena R Jr, Musa L. Post-Hoc Analysis of the Departmental Final Examinations for Selected Gen., editors. Subjects for Arellano University, Ay 2010-11. 2011. Yusoff MSB. Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review. J Taibah Univ Med Sci. 2019;14(3):203–40. https://doi.org/10.1016/j.jtumed.2019.03.006 . Knorr M, Schwibbe A, Ehrhardt M, Lackamp J, Zimmermann S, Hampe W. Validity evidence for the Hamburg multiple mini-interview. BMC Med Educ. 2018;18(1):1–12. https://doi.org/10.1186/s12909-018-1208-0 . Duff RE, Katcher PA, Daniels RM, Ramaswamy V. The Multiple Mini Interview as a Dental School Admission Tool: Can It Predict Noncognitive Traits Associated with Professional Behaviours? J Dent Educ. 2020;84(4):478–85. https://doi.org/10.21815/JDE.019.184 . Ali S, Hashmi MSS, Umair M, Beg MA, Huda N. Multiple Mini-Interviews: Current Perspectives on Utility and Limitations. Adv Med Educ Pract. 2019;10:1031. https://doi.org/10.2147/AMEP.S181332 . Tables Tables 1 and 2 are available in the Supplementary Files section. Additional Declarations No competing interests reported. Supplementary Files Table1.pdf Table 1: Mean, SD of each item and Reliability of Each Station (N=678) Table2.pdf Table 2: Pearson Correlation of MDCAT and FSc. Scores with total MMI scores & Pearson Correlation between the total scores of each construct **. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed). Cite Share Download PDF Status: Under Review Version 1 posted Editorial decision: Revision requested 26 Apr, 2024 Reviews received at journal 13 Apr, 2024 Reviews received at journal 09 Apr, 2024 Reviewers agreed at journal 31 Mar, 2024 Reviewers agreed at journal 30 Mar, 2024 Reviewers invited by journal 30 Mar, 2024 Editor assigned by journal 24 Mar, 2024 Editor invited by journal 21 Mar, 2024 Submission checks completed at journal 21 Mar, 2024 First submitted to journal 15 Mar, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4110146","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":283167143,"identity":"b28b3d65-614b-47a1-ad00-b8a62257f5c9","order_by":0,"name":"Sundus Iftikhar","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAzElEQVRIiWNgGAWjYJACCQYGGx72BsZmZlK0pPHwHCNRy2EGnmMMzMRpkZ+Re/A2z5/zMjzyzc3GBTX3GHTbG/BrMbiRl2zN23abh4eNsTl5xrFiBrMzBwhokcgxk+ZtuM1jD9RymIctgcHsRgIhhwG18Pw5B7blMM8/oJb7Dwh45gZIC9sBiMN420C2ENBhcOaNseXctmSglsRmY96+BB6zM4Qc1p5jeOPNHzt7Hubjj6V5viXImR0/QMAadMBDovpRMApGwSgYBdgAAIsoOqk3cldGAAAAAElFTkSuQmCC","orcid":"","institution":"Shalamar Medical and Dental College","correspondingAuthor":true,"prefix":"","firstName":"Sundus","middleName":"","lastName":"Iftikhar","suffix":""},{"id":283167146,"identity":"e4114b75-73d6-41ca-9006-731ab9c2eccd","order_by":1,"name":"Syed Hasan Shoaib","email":"","orcid":"","institution":"Shalamar Medical and Dental College","correspondingAuthor":false,"prefix":"","firstName":"Syed","middleName":"Hasan","lastName":"Shoaib","suffix":""},{"id":283167147,"identity":"8b5f1565-f8de-41f4-a605-7158ea9a322c","order_by":2,"name":"Shaur Sarfaraz","email":"","orcid":"","institution":"Altamash Institute of Dental Medicine","correspondingAuthor":false,"prefix":"","firstName":"Shaur","middleName":"","lastName":"Sarfaraz","suffix":""},{"id":283167149,"identity":"1f24c18f-6a4c-4575-aef9-02152f82c00b","order_by":3,"name":"Syed Kauser Ali","email":"","orcid":"","institution":"Institute of Medical Education, Jinnah Sindh Medical University","correspondingAuthor":false,"prefix":"","firstName":"Syed","middleName":"Kauser","lastName":"Ali","suffix":""}],"badges":[],"createdAt":"2024-03-15 20:14:21","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4110146/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4110146/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":53365780,"identity":"48162d5c-daed-4fe7-951b-c0716bfc1d25","added_by":"auto","created_at":"2024-03-25 06:20:28","extension":"jpeg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":46905,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eComparison between Difficulty and Discrimination index of each MMI station\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"Fig1.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4110146/v1/6ee25034b94232c9092ad89f.jpeg"},{"id":53365930,"identity":"22af7a33-6583-4735-85d8-eebba0edc4b7","added_by":"auto","created_at":"2024-03-25 06:28:28","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":277361,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4110146/v1/47bd6b2c-f23a-441a-bcf8-43e866323fcd.pdf"},{"id":53365782,"identity":"51851c6e-a074-4b8d-948b-9f27dd810103","added_by":"auto","created_at":"2024-03-25 06:20:29","extension":"pdf","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":291828,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable 1: Mean, SD of each item and Reliability of Each Station (N=678)\u003c/strong\u003e\u003c/p\u003e","description":"","filename":"Table1.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4110146/v1/a004b52115599d1bcf9bace6.pdf"},{"id":53365781,"identity":"6c3881ab-7a0f-4263-99d9-998d873f1ec5","added_by":"auto","created_at":"2024-03-25 06:20:28","extension":"pdf","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":192243,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eTable 2: Pearson Correlation of MDCAT and FSc. Scores with total MMI scores \u0026amp;\u003c/strong\u003e \u003cstrong\u003ePearson Correlation between the total scores of each construct\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e**. Correlation is significant at the 0.01 level (2-tailed).\u003c/p\u003e\n\u003cp\u003e*. Correlation is significant at the 0.05 level (2-tailed).\u003c/p\u003e","description":"","filename":"Table2.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4110146/v1/9ce9e11107d4af85dc3ff070.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program","fulltext":[{"header":"Background","content":"\u003cp\u003eIn 2020\u0026ndash;2021 Pakistan Medical Commission (PMC) changed the centralised admission policy of medical college admissions and allowed individual medical colleges to give 20% marks to candidates based on their admission tests or interviews (\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eShalamar Medical \u0026amp; Dental College chose Multiple Mini Interviews (MMI) for this purpose. In MMI the interviewee has to rotate through more than one station with the different interviewers at each \u0026lsquo;station\u0026rsquo;. Different tasks are posed at each \u0026lsquo;station\u0026rsquo;. These may involve answering questions based on scenarios, completing a practical task, or participating in a role play. The MMI allow the interviewers to assess the non-cognitive skills of the interviewee (\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e). This type of interview practice is being adopted widely due to high reliability of its score reported in literature ranging from α 0.7 to α 0.98(\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e)(\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eSince health care is now more focused on patient-centered approach therefore it is necessary that apart from knowledge, a doctor must also possess interpersonal skills, communication skills, empathy, and integrity in order to become a successful practitioner (\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e). Every medical college aims to select the best candidates for their MBBS programme as in a few years\u0026rsquo; time; the healthcare of the country will depend on their competency. Nenad (2020) concluded that programs that take into account both academic and non-academic performances of the candidate are more likely to enroll students with better soft skills who otherwise would not have been shortlisted (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e). Studies have reported that performance in MMI predicts future academic performance of the student in medical school (\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e)(\u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eShalamar Medical and Dental College (SMDC) has been conducting MMI for admission in its Bachelor of Medicine and Bachelor of Surgery (MBBS) program from 2012\u0026ndash;2016. The MMI was then discontinued due to the introduction of centralised admission policy by the Pakistan Medical \u0026amp; Dental Council (PM\u0026amp;DC). The reliability of the last face-to-face MMI was found to be acceptable reliability with a G coefficient of 0.70 (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eSMDC decided to conduct MMI again as the new PMC policy allowed medical colleges to give 20% marks based on their interviews. By giving 20% weightage to individual colleges, PMC placed a lot of responsibility on the shoulders of medical colleges. For this purpose, MMI are considered a better option than traditional unstructured interviews. However, in 2020 and 2021, COVID-19 pandemic made it impossible to conduct the MMI face-to-face so SMDC shifted the entire process online and conducted the MMI through WhatsApp video calls. Conducting MMI virtually was found to have several advantages like, recording of the interviews, ease in accessing the faculty and less expense to the candidates as they can take the interview from anywhere in the world (\u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e)(\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e). In 2011, a study conducted in Australia concluded that there was no significant difference in MMI conducted remotely or face-to-face (\u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e13\u003c/span\u003e). However, apart from logistic and cost issues, it is important to determine the reliability of online MMI for their continued use.\u003c/p\u003e \u003cp\u003eTherefore, the objective of the study is to determine the reliability and quality (difficulty and discrimination indices) metrics of MMI scores conducted through WhatsApp Video call.\u003c/p\u003e"},{"header":"Material and Methods","content":"\u003cp\u003eThis descriptive study was conducted at Shalamar Medical and Dental College (SMDC). Ethical approval was granted by the Institutional Review Board (IRB) Shalamar Medical and Dental College (REF: SMDC-IRB/AL/44/2022) IRB number: 0484. Online MMI was conducted at SMDC admission for the session of 2021\u0026ndash;2022. This interview process consisted of six stations. Each station was of 12 minutes where 3 minutes were given to the student to read the scenario. The student then answered the questions posed by the interviewer related to that scenario.\u003c/p\u003e\n\u003cp\u003eThe constructs to be assessed in the MMI at SMDC were selected from those that had been identified in an earlier study (\u003cspan class=\"CitationRef\"\u003e14\u003c/span\u003e). These included \u0026ldquo;communication skills, motivation to address societal needs, ethics, empathy, critical thinking and leadership with decision making skills\u0026rdquo;. The constructs were shortlisted by the medical educationists after thorough literature search and were finalised by a committee headed by the Principal, SMDC after giving due consideration to the practicality of assessing these constructs in online format.\u003c/p\u003e\n\u003cp\u003eAround 137 faculty members were trained to conduct the interviews. The training took part in two phases. In the first phase the faculty was acquainted with the philosophy and working of MMI. In the next phase the IT department trained the faculty in using WhatsApp to conduct the interviews.\u003c/p\u003e\n\u003cp\u003eThe students were informed about the interview process through detailed explanation on the website. A video was developed and uploaded on the College\u0026rsquo;s website to inform the students regarding the interview process. The faculty was provided with the roll numbers of the students that they had to interview with a corresponding assigned time at which they were supposed to contact the student.\u003c/p\u003e\n\u003cp\u003eOn the day of the interview each faculty member signed into software developed in house by the IT department of SMDC that connected the roll number of the student to WhatsApp directly on video call without the need for the faculty to dial or save each number separately. Once connected, the faculty shared the scenario with the student and started the interview. At the end of the interview the faculty entered the scores in the same software.\u003c/p\u003e\n\u003cp\u003eEach interviewer scored the interviewee on 5 criteria using a 10-point scale (supplemental file A). Each criterion was pertinent to the construct being tested.\u003c/p\u003e\n\u003cp\u003eEach student was scored out of a total of 240 marks, 40 marks for each station. Hence, the student rotated through 6 different stations with 6 different interviewers.\u003c/p\u003e\n\u003cp\u003eA total of 731 candidates participated in MMI. The participation rate was 95% as 678 candidates appeared in all the stations of MMI. As Universal sampling was done, the sample included all the students who appeared in MMI of SMDC in the session of 2021\u0026ndash;2022. The participants who appeared in all six stations were included in the study while the candidates who had not appeared in any one of the six stations were excluded from the study. This study was conducted from December 2022 till March 2023. The scores from each station were compiled and data obtained was then entered and analyzed descriptively using SPSS version 20. The information was kept strictly confidential. Serial numbers were used instead of names to protect the identity of the candidates. Each station\u0026rsquo;s scores mean and standard deviation was determined and compared. Overall reliability of the scores as well as reliability of each station was calculated using Cronbach\u0026rsquo;s alpha. Correlation of MMI scores with MDCAT and F.S.C scores and between stations was also analyzed.\u003c/p\u003e\n\u003cp\u003eItem analysis was employed to assess quality of MMI stations. The quality measures involved calculating the difficulty (D) and discrimination (R) indices of each of the stations. Difficulty index (D) was calculated as such:\u003c/p\u003e\n\u003cp\u003e\u003cimg src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA6wAAACCCAYAAABCdUWfAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAAFiUAABYlAUlSJPAAAABhaVRYdFNuaXBNZXRhZGF0YQAAAAAAeyJjbGlwUG9pbnRzIjpbeyJ4IjowLCJ5IjowfSx7IngiOjk0MCwieSI6MH0seyJ4Ijo5NDAsInkiOjEzMX0seyJ4IjowLCJ5IjoxMzF9XX1JHsXXAAA3j0lEQVR4Xu2d68tuVfnv77Xf7RdWWuwX8SvampBU5FbLsBISSjsgxc5MixCScpkIWZktlV6k5TLrRdiRgojKLKOI7AgGGUF5+CkUBppESC82dtD+gLXnZ8zxve/rGY457znvw7Pm8zzfj441xzOO17jGNcYcY57uQ8caZsYYY4wxxhhjzMT4H/lojDHGGGOMMcZMCm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9YjTHGGGOMMcZMEm9Yzcp897vfnb3yla+cHTp0KLl3vetds9/+9rc5dv/wj3/8I7X1xS9+8ezWW2/NoQuIo/3XX399DhnG448/nso76aST9qXetgX9gd2hN3Q4hocffnh25ZVXpv7aLX7605/OLrjgguTMchgLu91H5uDBPML8y/g0xx/mcsY98zpjX+db+ul4g2yc38eeb6bKbpwHtW7iXF1bNxkzmmMDOXr06DGSL3MnnnjisfPPP//Y4cOHj91xxx3HnnzyyVzC/qWmG/Twl7/8Jaeog57KfDjCpw79e/HFF6f+xeGX/Pfee29Otff58pe/nPpSbaOvS2I/DgX9KQ9uP+ls26Ar6a3WHzUYi+g89uW2ufvuu3f0814Y18eThx56aMc8sht9ZI4dO+WUU3bovMt1zVGc51UGx2VjknER0/P3EBjDjF/sZF0og/FYK0t2qLmCI+N42fmc8yBtP+uss+Y6w0/YQVgHrQM6R8+yMWxC+l9l3pStLMurPlNd9FeXPVIm5Q211ymiNsg+cduAdVOsZ+h52pg+Rlsrhigj1MkmTsZMOEeOHJlPABw5oe130AHtVLtxTH5DTlQMZtJrAzh1aCfyxgUMcjNB0f6uhc1ehHbh1K+1iVf6GDMpUyZ6Il+pS9MPumNsMf8sW0SWqK9wu4U2rcsWT6ZF8+Fu9tFBhY2CdN3nmP9q6CKQ5i+O/N117tP6QXOlxsaQTQDjZ8wc24U2R7XNapwfSteVBzQnkSa2BX+fPkwLc3k5P2o9xbpoLNos9c256jOcziOyT+yyBnmQCdn2IrLB3ZpjdWFqE+PWmJWsVYbeNxkwATARKG3XBLDf0MlwTLuZREi7VzYtmoQOEjoBbnrilZ1squ8PyjhbBguKmk4Jk843TZfutTjomy93iy69rMsm7Y5F/rb6yOyEfmM+l12UTn1R618uTBPHIj9CPsLLjYYW+9QnWCeQNobVYAxtYvxok1Kbx5EF+WirNqak1/jFEa9Ff4Q8xNc2tNLHJsfItkAHmz7HLQPbQz+bqjf2V5/NaH1a9pnyl3Yt1J8c9ypqA26bbGvdZA4mK1mrjHDZCUQnBw2MvXpVaizSj1zXxBch3V6YAOlTtesgsa2JV7rcRN9r8Wfaiyo1nW7rRN2ney2Als2Xu0GXXtZh03a3rT4yzwSbrG3AhDYT5R1Q9TkbuBrYGfHR1rT5LceB0nbdvdRFYOpcFzaNXZtOztNd52rCkRFXS6O4LpbFTwX0s9ubC1342ES9shVdQOiac2XXtbu30ba7xgblYrd9Y2fK7NYci56owxtWswm2+tGl5z73ubOvfe1r+a9Zesl7Ci/Q7wbNRDhrJrzkv+KKK/bNR3UeeeSR7DNTg/FlZukDD82iI/+1O+wF3W9LL7a7vQkfG2oWkuk83cVvfvObdB5785vfnENabrvttnR84xvfmI4lnP/g9ttvT0f44x//mH074eM68J///CcdI6wXLr/88tm3vvWt2cknn5xDV4Nz8Je+9KX0ccBam/kQzQc+8IH8104IbzYoyf/DH/4wHWv0rW+0Hpgq0s9u88ADD2TfeshWvvjFL85e9KIX5dA6N9xwQzq+/vWvT8cIdnbWWWfN/vWvf83uuuuuHLqTq6++Os2lX//613OIMWbbbP0rwa94xStmhw8fTv6+CWC/ccYZZ6STrLjwwgv3zRfmzPRg0/CLX/wi/3Vw4auEH/vYx/Jfu8Ne0P229GK727uwCeX83Ad2wwavhHDgPFfjhS98YTreeeeda12k/tznPpe+MlpumFfhpptuSse3vvWt6VjCRqcPbaxraDN64403pmNEXyHWOmiKsFlnjbKXwVa4qFCz1wht1YW7l770pelYoosT8YZLRPZ4yy23HJibMMYcb3blZ23iCaJrAtiPMKlxBRvYrHPVedXJjZMeEzGfIdcn31ksrrMJHlMmP8lBmte97nU5ZDbPhxtzB5m01KvP13Psqhd98Tl5ZFNdkrOmS8omjnRAGu4sqS7awQmrC2Qgv9JTV98n2Smfxduyny1Rm9UGFmHLfk7hK1/5yo6fDUImyojyIy8yxivjSo8rKeVQ+7p0GeunfaQl/xCULzrK7IrDCdLF8Khb2o9NoA+VB6S55JJL8l+zZKvKH9NFxtpHZKzuhWxM6bpsWWBfsR/wa8MwhDF6oe3R/mVzZboxbR9ix5uCupAr1oOtdI1hjV3J1jcegPByPiJ/V3+gJ6VHh+RXfRwjssVYdk33Ymxbx8L8xHmr3OAhD+GgjWlJ3AjoqZyutI899lj27YT62fB+8pOfzCGrQz/owsq6m9/zzjsv+xZcd9116ch4YPwI+vQTn/hEumN3zTXX5NBxlOMHf81GFS8X7QB/jIu2R9zpp58+71MubNXKWMaYsYENKY36JdZbjo1lyFa+8IUv5JBu4gW25z//+dm3E12Iuf/++9Oxxvnnn7/STZhy3aX21tYD6FTjXH2B7khPPsb9sj4qzx/U/ac//SnHjqPWx/iXncMEsihvbFMXpa5UF+M5QjlKE11ZfhlfUuoKf9fcbo4D+dHgUei59K73A0r0ToDcMuLz9WPdFJ6VRy9RDr1PgcNfg7iud8vIw/sUegeY9yYonzyEd+XrY9Uy1333gfooX+8BUa/0Qzi2IojjPZEoD/F6LzraH+Fqk+QjP2kpQzaLI6wGdZCf91okh77yqLyxX4kjreK6xoPefyIvMuHiO1G4Ut8qt0tP/B0Z0i+8K0Tb9a5Y1CXHWKbKi+9rqe/GjHuVj5NOIbYHF/UqSE8c6SQbfvpT+Uq9QV9c1BNlIh9uiH10sUz3GlfUoXes8Efd0DclyEc62izdUZfskfAxqK6aXkDvdclOFab6oi2IZW0fa8ewrMwukJsy1T7Klu5r9iVbUvpyPJSo7+L8QF7lITxCn0ZbZb6gP2v2S9mEoyfpBL/SaZ4WY9u6CuiHOkpkJ1H+EsKVJvY95dFOoTFe1qO0mqvWRbqMdY9F/ay+L4njGb/sCbtQn46BPOTHSc+UqTGlOiJR76UdUJ7yYoclhNXyDWHs2IisUy/IVqItaizU2qn5B9eF8uO6bFDv3tLGocgOkUE2Ecd5bAPjTPrDIZPGXjmPlO+YC/SObjR/0Dex/bihIC91Rl3LxilnmU2V9coRXoPwKDv1q1/K/oZyrSb9liAvLsbjR1bqjPar8rpkNLvLuBVBRkZYM9AuZES4cpItwXgwllXcsrJ3A/TCwBK0J048GoARwpG/RAO0licO3jHtXqdMZCQeNxYmFPJpASM4IajMODFIltLOVA6ulJE2KY58UadR9lLXlEOb6aeSmA+ZSvrGg2StbUjUPlyXnBHsSOFl33XlEaTnZBMnaYhlRt1zoqu1h3Jq4V1Eucq+om6dEGonXPVJKXMsM+pN9MXFvGPso4+Yr4b6Gf1TZ9RDtIES+qO20Iv2XxvDXShPrW1qQ+3EHNtX9lNf27vi+uwY+srsA1upjU/aVIZrsVkuRuPiMcomW61teKIdl+M8tpW+pD7CkCempdxyXgTJiYt2M6atq0IdNXugfMlEX9WIfRjlUV61lfL5u7QDxsmm2gGURz1j5q4Iuid/TR+CfqWPSSe3ThskczlvguJq5ytsqatu6b+mB5U5VuZVx4ZYtV6Bzsu8Q9qJ60L5cV02HtMMpatMrQ9r9qX+JA3x6FSoLbV2au6onVuH6KCkS6fxfFTaquqhDfST4ml/XBN39V/t/KA47KqsL/ZJ1FMEWcp5H71u8lxrtsO4FUGmb5B0oU7HdU0A+wX0Ug5ABpYmblw5YGp6IY/S14gng9pgq7FumcjYl78PysXVkE3FRZsmi9qJTjKUOovy1SYsta3sH9Vfm9yhbxGgE0M5HqIua7LEvojtkK3UFiOSs5RjWb8g/7IFQ+wbwvi7Jnffoq2GTkw13VEWcTVbI32tLmRSW8v+h764qCf0XKL+qm0eulime9pBHHoo9Rkv1sQ5QbbRZY/KM3Tcg/LU9CL7rsVBzUagr+2r2DEs02cXpKfcEuSI9UjnNd2RFl0gd9SF7LQmL6iPcbW5vS8vCyHia2Mt6iLaJH8PaeuqaO6t2UNsay0eotylPLRD9saxtHHiy7ahG/UBfcNcVtNXFxrXY+cuQT7KQL99IJPaJrfKQlc2Uetj6LIL6Btb6rtauX35+lhnbMCq9QJtr80vQ9qJ6yLKPMTGu9KUIGvNjobIWzt/037JEJFsNd2A7KvM18cq6zHJXht3caxwFOhG5dUgn8ZzOYfHuNq4ow2ljlXfJs+1ZjvsyjusJaeddlr21eFZeJ7TX8WVz7ZPBb489+tf/zr/1X6dbtkz/7/61a/SsRlg6VjClw55vh+GftxiG2UOgXcReN/jVa96VQ7Zyc9//nNmpx1faeRdI8JuvvnmHDKO2pcga/XH95vOPvvsdCzh3Ykunv3sZ2ffTtA1bW5OGlVZur56Sfg///nP2X333ZdD1oNx0UzKs0996lM73t+QU9uRVePnOc95Tvqbd2XIH1n2cZISvtwIX/3qV9OxBrZWjl3S177aWdPlKtT0L/t46qmn0nGTIHcpe/zoTfxKqt6Lestb3lLtM9H3jtVQeJcU+4DXvva16Vhy2WWXpSM2UXvXqsam7XgZzUIl2TLzV7Ql5Lj22mvzX7PZ9773vXSsfTCItLxTidxRF3qP6ZxzzknHkne84x3Ztyi/pCvvN77xjXR83vOe94x+jt8MiO/8Dm3rqvzkJz9JH57psod1YEyjY+Z2jvGdUtr48Y9/fMccwzmIefmXv/xlslPOo7wrOuY9R+wWln09tgb6pT5kqs0ZAtmRiY87NYvl+YeYeH987HtwP/rRj9Kx9r4s0C/0Dxzv74JsYmysAvr+zGc+k84dx5unn346+/phLmRu6bOjLmrrjK6PRn3nO99Jx66veP/Xf/1X9g1nnfVYbdxxLvzoRz+a/IxrzW/rrFFjnL4EHfn85z8/P5eJ3TzXmvXYlQ1ruRAtF20lfKSBE/UqbspfIWZxyokMtBnoo+/z+SIueIf85Mw2yhxC108ajAE70gcINokmSFhmm2PQomPdMpmQWRRgL/GDEUPRBxaOHj2aTjh9TidSFo0suJikGVe1jetQtGDhpBQ3O/Qni9CL809gxJ8IIB31L/uK6X7lwQcfTMd777232k9yXR+sGcMQm4oXGVcdy+va8TK0yWERw2K+9nEOGPszGti9NjxdMG60QRlb/h/+8Id0rPVvdHETN7Stq0I/aVyWdH08qYuh6bEP/SxJXNAzLzB3sLglnDmBRSlz09iN4Cqgh8OHD88XwjWQnYvQbBBIh2NjveqmdcgmjAuhcDwX0tseG31gK1wciLYyhLHpuz7MFFl1TuQ8x9jd9BfcZWtdF9M3wSbWY3HDrQu2665RP/KRj6Rjbb3B+bIcx7t5rjXrsSsb1j//+c/Z133VJMLVw5rBDHGbuLq8TRgsnPyAEw2T1Tp0XWFbh22UuQ6cFNEbJ+i//e1vaZOzSbZxNw3+/e9/Z99qMMFiH6eeemr6PUSuGA4ZPyVq35h2ckLAPmWrbDC0cR27MGbDrnK++c1vpiOwEGXB8d73vjf9Hb84S7oPf/jD+a+Dh2xn6JX7bbPOhYNN2fEymCNYdKhs7InNHF+1jFfht0XX0yPL0IJ/jIzbbCuLPGR65zvfmUN2MuTuTLTboXdzun6W5Pvf/346xvOS7FEXBbcFdsv8FS8W1Hj3u9+ddPa+970vh7Qyxk0rZW3SDrt+UmiKrDo2umBNwPmJCwG1u2LaAHLeUpguuA65yx7PlWM3uMvABvh6LRs97vide+6581+T2BTLLiKswybXY6s+wdG3RqW/dLEN/Qp+O/r9739//mvB1M61pptd2bDyeJEob8cfRDj56eooC41lV151BX4ZJ5xwQvYtZxtlDmHsnRUWYGyUODk/+uij6XGUTZ9Apgg2gY3w+BAnZmxm3cfzxj46hZ6plyuVOgHQf8g1duF16aWXpiMysIEhP35OfDxqxEKVkyztJp6T4Bve8IaU5yAT587dID522sWYq/bbsOM+KJvXC9jMaY7lUXg2FCX33HNP9g1nyM9B8Dj9KsQnPYYwpq1j0OPAXRcpYv/97ne/y76dxDtOQ/qbRTDzQe1nSbCbEi1Y170o2Ae2yyOc3/72t3NIHeYrzYvluUmbVmB+G/vEku7+9KEN8fFmm2Njk8RHl7WJLdGd4KEX1l72spdlXz/MrzzezkUYXgVg/JYXaKbMptdjcR1RrjXXWaNeddVV6ci41HqD8RwvKJXs9rnWjGfrG1YGqO6c1K6eHlSYqNAHxN9IjJx55pnpyImOQddH3wIjso0yhxAndCaOLuIdZyZHFmBHjhxJd843+bhuJC7A4yMk66KTM5PmmA0eMmAT6B9drbtB1wm6fESmpOs30agfOe6+++60OMJ24uO7Q2DRKntnYU7+eLVTG+LPfvaz6bF+5olt9fde4Lz87hp67xuny37HbghxbA5ZUA+9kLBpOx4D9sZmQ3cuGINanGoO1GKmBuOV+Qfi49BDNvRve9vbsm8YWhRj+10gD4/f1ehr6yrQV12PAws9MdG1ofrrX/+ajkrXB2278MIL0wJ+W2Nem7ohG0BAB/QH5+llMv39739Px650nEOlh6G/f6kLELXNesnxXFNte2x0gc3XnrCT01hgbClMF07iuairP/T459AbLM961rOyrxvmGh4b5zyMXUmebSB719MJm2Ab6zGdb+JacxNr1NjH3FntW1Ps5rnWrMdWN6yciPTBFdDHJUx7cvvBD37Qe3U0Xg3qejdXV7hrjzrU2EaZQ4gfM2JTWju5MWFEmBzhTW96Uzpui7gAj4+QrEs8OY/Z4EmGiy66KB3Xhclbdvae97ynqnvC4h0nLdYj3AldtlDtQx9Y4EMZfFAp2qL8LNBuueWW6seWDhLaxHDSZvNQu+DBeNnE4+y6ww1dc7Q2QMg1dOO5aTteRs1mWVhp8a/F6Wte85p0hC7d3njjjbOXv/zlyc9cLbtH57X0wEaRcTb2yYC3v/3t6Yjtxwt2EeR5wQtekP8a3taxLHscWOiJia7HARWudH188IMfnF133XWdC3i1KTK2ffr4zJA7svQxj613bVbp/6h/3eHpuyOkx1Bf/epXp+My9DoEm5uuiw+ar4foWGi+wFY3wbbHxrbQ2iZ+CFOwaUHvy+SNNrjsQ6LAeouxxRxas6tNoosYzCnrXLyKbGM9VltrbmqNyutGwA0z1hxda4rdPNeaNTm2AmTDNR2dQ55JM+DTJ7WVtvaJ6f0Kejk68BPt8fPi91Y+jc4nxIlrJs/qJ/WbhWZyYz7xv06ZyCh5x9QJ6ER5qZu/KY/PiTcnvWfIo7TEReKn3MlPHn2SPMpXg74hruwf6lC+sj7KRx+1OJA+a+NB+XDlGIiyxjjJSN4I6dERcciP/pUvliUd8hMC8kfd45CZPDj8lItfIENtzMpey59SGALyqv6aHtVu5o0+YjlRZqE4yU96peOo+Bpd9tFHLLNP9zX7AOUt2yJZcPQPOidNHC9jxqDKqumFMhWvsRTR+Ch/lqKv7ZJ/jB2DbAw3Bsqs/WxGTfaoW+SjTuTiSFxpg7RHMjNeSiRzbVyontjGCO1X2TjqLuXBRca0dQzkL/urC817ZbukC8paBvpaNt41fqJu1c6h8xDpSL+sbcjepVsgnL4o2yx76pKHepe1s4T0lFnLhz0S1yywc8gCwtTWOD+gR5WJK+cOtSH2W20uKFlnbIDqreVdlWVzLkgXmreEbKvs4xLVMbRflb6sE79kkbxRV9IP+Uvi/BuRfeBq9twlSx9KX45rjS0c8lCe7KavbzXv1fSnuYX4mnzYdmnfNWSXfXYAklN10ibasuq51myH0RvWaJwYWjkQ6GSMLRrKqifPvYgmEAbT0IlAkwd5SxgkGkyUqTSUzYkJPY/V76plxny4cuIagk4GpaPO8gQR0+JHT9hcXFwjPzJpMtHJGleWR5sURzlxAsKvk4biqY96KT/GESadkQ8ZCKcNZZ9Tp8YCDvkol2NsH2kYN+TXCR5H3aSPR6VHpthPqodwyqKOSKyvdOViQvVQp9pEm2kr5UfdjUEylHoCtXvZQiHOQbUToWwAfSC/5MVFHRAXifZB+8e0sUv3tJMwlVuO8b62UH/MGx31lWUto0svgvoVL3sgnrSEd/VLV9tXsWPqk5y4ZbYQUXnkUbskA/0e6dMtNh7HlaAstZV2xjoIL+uA8nxZs3so54nokEd1CcKHtnUMlFnaYRfSIXlki9LFkPFDm2lbl06E6lFa6YqwZXUI8kmfXXmkvyGuLEMyEkefKx5Z0UWXTfUh/VAmZUhP6Jq6utpPvORUX3CkfRrLOMrmb1HOReQbOseo35VXcim8yybjuZx0Y3XUhdpJG7qIdoRukVn5howByiat5splRBuU7tEL8x31KQ550BsgYwwv+7tvruRvxakPqJNy8CuutIMuYp5YVt96LNoUeUobjmkjhEm/lCk7JD9tHmor6s9SNyXUhzySNTrqGjoOzHYZvGFVxy9zdC6GxgA8SJ1MW2v6wA2BQdinLwZcHFCaZGqDfShjylzW/2OI9WIvtL02+SBHnCRJpwlPE7wmvGX61+RXOtolKIe/0QNxUR/kx+nKIXTpJJYJyEw7aCvxtF0TKH8TV7afMmJ62YZODshW5iFOslNmrR/LPscf2yTQNe1VeapT+lgV2oFsXVBHX/mSpXQR9KL+5sjfq9pH35iM1HTfZR/UBbU4XETlxH7oGi/LqOmlhPZSvurCBmmPxl2NPrsbY8fL+mgZlEW7VB8ujrWSUrccu8aNKMcyDn3V7ETxpaPOGmXZ0n1NnrFtHYL6ZIxt1XQ4VIYx8lIPulBbl/VTDdl1bb5T24c4yqlR6gJHGwkbK6voKnOZ3ohXHtKrzSorbqoFf0vHpKnpqY8xYwOUpuaQcx3ITzmaa7soZSZ9l7wlyjOmb9Gp+kX9AMhBeTj1LbKQrnS0behcSbo4n6s/cNRFWdQ9hGgfKkt5y/VYhDTEq91KN2Tskwb7VT7KQOahOqdu8gyBMik7ykkbVznXmu1wiH+ajjHGGGOM2ZfwLh9fOG0W3Ut/psaYPvjmw+mnn54+7jT1n1I0Zr+wKz9rY4wxxhhzvOCjTkeOHOn9OJAxQ+CL3SeeeOKODwQZY7aL77AaY4wxZt/DRvWCCy5IXw3mNySNGQtfET7llFPS7x9v86dpjDE78YbVGGOMMQcCNhxnnXXW7Mc//rE3HGYUuuDBzw4dz9+/NeYg4g2rMcYYYw4MbDze/e53p99b9abVDAGb4feQzz33XG9WjTkOeMNqjDHGmAPHrbfeOjvnnHO8aTW9cFf+tttum33kIx+ZnXzyyTnUGLObeMNqjDHGGGOMMWaS+CvBxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmiTesxhhjjDHGGGMmyaFjDdlvzFr84IH/N/vhfz85O3So+aOxKgwr+Tk2/7UhhB2aYXYcWxp/im/Tp1TNP/O8KbDN2wbmvDnhopwFOVmOO5aSSoZ5+ixDisvpQdGkJ2eZHjjsEIljLr/9o/WrXcmfMjVeBTXI25bb1NbEyz9Pn/1F1ias+Senb1tIfMgLbYHJq/Rzf2IROK831zJv+6KIhHIofRvf+jm2elnIELLOM8/z5qBER3ltUCgvJ5Mfb9R9myfLkv6eR7V+js1/c52F9C1tiRCD5v6cvvUS2OTN6Tt1lrMofQpLeRZyRhlCMhLmQ8i7KDD9ncrJCRUUo+fpsz/JmdOnvDkh/86rzX+0cfPQxi3SR5lj5kXQOD3HNqbAIv1cZgU1yKtko9ud0+daQ3n4mlAisl/p5/4CZVHelDWVGNO3MsyLhZSn9ZJOp2WVE2VOCUN6mHtz+ja+9adjClrIME/fkKJD+jasDeTYhixkyEW0SbM/pm9ZRMaguT+nb70ELsqfyxnTN7SpFunb+NbPMZUWZFCyRBuU62gD2zpSIenvVE6OVxnxPBXLS1k5Vsrj2IaovB1VkCnRltsENP/P/akMYkO96d82eJEsewhr/ot9W+qAJsqfjqnEIGdHG1VxisuFtOnw50jV2wTlInbEztOn+CAnieQvUBblbctr/fP0Qc9KD4pOcYXM6ouUmLB8EHNvTt/Gt/50TEELGebpG1J0R3lJljZzzrsoLwfiaQ8pbB7UENJD6Ksc1XrnQW0gwW0dIT3lNP+1cW0If8zzNp6lOmtQiDG7jTesZmNow2qMMcYYY/YveV8bPGxoW782ylu/6FNssueR5M2ZFiGkaz2L9MS3FxWUPm70OWiXpOgosxLE9IpSxbU2xoSKVxlJlhSWk0HOQo55GfLn9G1ILienL1H6Nj77c3r+acOaoPRvG7xIdmj29jOeNzvjhSekuN3GG1azMbxhNcYYY4wxZv/x9v/zvNn/PfN/5b92F29YzcZJFnUo/T+/MpUfUmn9TSBXaham117dAoJSquafed7kiQXmvDlhzYSVRXlT1izDPH2WIRSbUHS8ytXWt8ib/5z70zGX3/7R+tWu5E+ZGq+CGuRty21qa+Lln6fP/iJrE9b8k9PP5UwpFnLmApNX6ef+xCJwXm+uJV2xI2xRREI5lL6Nb/0cW70sZAhZ55nneVPQPLA9hvh0TEGhvJxMfrxR922eLEv6ex7V+jk2/3X1rUqEGDT35/Stl8Amb07fqbOcRelTWMqzkDPKEJKRMB9C3kWB6e9UTk6ooBg9T5/9Sc6cPuXNCfl3Xm3+o42bhzZukT7KHDMvgtpAglO6mL4NTb4yffvHM9PPZVZQg7xKNrrdOX2uNZSHrwklIvuVfu4vUBblTVlTiTF9K8O8WEh5Wi/pdJVb5USZU8KQHubenL6Nb/3pmIIWMszTN6TokL4NawM5tiELGXIRbdLsj+lbFpExaO7P6VsvgYvy53LG9A1tqkX6Nr71c0ylBRmULNEG5TrawLaOVEj6O5WTEyoonqdieSkrx0p5c1kq5eVMibbcJqD5f+5PZRAb6k3/tsGLZNlDWPNf7NtSBzRR/nRMJYbAjjaq4hSXC1nUmyNVbxOUi9gRO0+f4oOcJJK/QFmUty2v9c/TBz0rPSg6xRUyqy9SYsLyQcy9OX0b3/rTMQUtZJinb0jRIX0KS/+2YakVZXnQZkrekFVBDYvAlCe3uw1oXJFeeiL4Gemb0NZWFun5Y5638SzVWYNCzMHEG1ZjjDHGGGPMniDva4OHDW3r19Zi6xd9ik32PJK8OdMihHStZ5Ge+PaigtLHjT6HXEWbl2MqMQSG9IRL5sVhkT6lo8CYvigjyZIzh2TJTynzMuTP6dsQlRfyBpS+jc/+nJ5/2rAmKP3bBi+SHZv97+f9zxR+PPCG1RhjjDHGGGPMJPHP2hhjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjjDHGGGOMmSTesBpjzHHgH//4x+wrX/nK7MUvfvHs1ltvzaEHh4cffnh25ZVXzk466aQcsv/57W9/O3vXu941O3ToUHKvfOUrZ9/97ndz7P7ipz/96eyCCy5IroS+p99p/xgYM+jroI4Zs10ef/zxZFfYJmN1HcjP/LaXuP7669dutzHbwhtWY8yeQIv8VdzUTsIsis4+++zZFVdcMfvLX/6SQw8GLApZyL3+9a+ffelLX5r961//yjH7GzZal1122exTn/rU7NixY7OjR4/O7r///tkll1yyrzZfjDU2lG95y1tmv/jFL3LoTgin32n/0LHJxZ1TTz016eugjRmzfZiTTjnllNnHPvaxteckxvN3vvOd2Re/+MUc0g3zIRtFLt7ofFXCeIrnsy5XG0uEcZGMTThpOPJ3Le0111yT5PbFIDNFvGE1xuwZzj///Nndd989u/fee+furLPOyrGzHeGku/jii3PMtLj22mtnjz32WP7rYHHyySenhdyQxdx+gUUpG633v//9qf2ADRw5ciT5n/3sZ6fjfuC0005Lts1Y7YK4E088MR1f+9rX5tB+3vGOd8weffTRlM+YTfPJT34ynTfWhY3vgw8+uHR+42kBNqpsku+8887ZRRddlOrnYlaEpxGGXKBhXJRjiYtkr3vd69ImlYtDlM2Rvwkvn+547nOfm+S+55579tzdYXMAaAzYGGMmT3Niz76dNItezvDJ1Wg2rceahUD+az2ahcOxo0eP5r/Wp9lsJ7k3WeZegT7p67f9xOHDh1M7N2WHewFsmjYzPjeJxvumxsymx7TZ22hOWmWs3nHHHceajeOxJ598MofUeeihh+ZzP3NDX3riOfdRNjKV7u67756XE8GuCef8V0PjiHQlyEM7PC7MlPAdVmPM5OEOFXenVuGqq66aPfHEE/mv9bjtttuybzNwRdvsf+67777sM1Nj02PaHEz0FMV1113XO69zx5TXIbjT2WxC0x3NvvSU+/vf/z49xssd1NI9/fTTKd1b3/rWdBR33XVXOp5xxhnpWHLeeeelo9JFkId28Hg08hozBbxhNcZMHh6j5BHKVeCkzsl+XXjnh3cujRkLi1MzPTymzabQI7Q8ut4FjwFffvnl6R1ZXgdYdl7iw2VHjx7t3dD+5je/SY8Dv/nNb84hO3nqqaeyr07X6wjve9/70pFNqzFTwBtWY4xZAleZL7zwwvyXMWav4zFtNgUXPviQGN9T0DvqNT73uc+li1e8t3rzzTfn0G7YhL7iFa/If9XhPdTaxlcbUS7I1O6Sfv/7308b3Te84Q05ZCdsknnHnHbVPtBkzG7jDasx5kDCV0fjlxnx83VEroJHCDv99NPnX47kirPyxK8p6iMa8YuO+LnyXpa5DpRV/hwOCwp+PoQ6+aBG+ZVHySMX4/HHuPJnSKiPRZH0AyyAWCQpT9lG0kkPHLlTsAwefaMclUn5fY+jKT3tJT1H/ia8RIs6tY2/SY8bIptAnrJOyq0t6KJeBR86UVjsg2WgW9KX9lp+NEWUti05a/pEX9gtadQOyh3af5JN6SkHHfXd2aE89XUXZf9S/jKdUW60Sxx9XspPOUPGNIyxM3QhXZIWmUnbZSNdqBzpNJYVx5mQPmXftfFZk3fVfII2xfTqo5qMY9tE2cShS0A25V2nj0QpO+OltJOh8GVd4MNJXSALXwiHm266KR3XBXmx4fJxYOBOLxtS4nkEOY59xjcbZx5H7ttg67Hh22+/PR2NOa7kd1mNMWZPoo9HDJ3O+KAEH7zA8dEK4MMTfJyCMgivfYii72MvlMmHMfhQRSxTH9Ygb42+Mmvw4Q2VqXyEqQ5kUBwf44ggV8wXQX61P8pK2ZJR+fhgCO0s69PHQ5BP8YrDlTqN8qjMmB5HGHElyEXdaiP16sNGMQ91qF04ZCp1GNvbh/SMDqhPYZL7y1/+cgqrobpkG2OQTpFTOqQc1XvkyJEUJtReyVPqRrIDeWMfUi5pSUd9qoNjbUzIxnFqW/ygDC7qlziVr/gaah9tUb30dcxX2jDtJVx2GMNwNd0jW60sMdTOBOXRdsnMUf1Rq78GdVAn5SuP9Ew5UZ/IJXkUh0xRT3KERV2uki+C7SCTdBBl5BjtbEybCC9thDpIG8PE2D4C2YXGMi7aCm5of4HkQpYu0JfKRqY49siPzFFnQ5CeukAnqhOHfJqzpK8+Yv6xshmzabxhNcbsaVjs6KQ6BKWvLcIUx+KoRHG1xS1hxMWFF8QTfl99XQvmLrTQQc5yoaMyS1lA+ca0AbSwZPFd1qfFofJqowSk02Ku3FixIFQ+ytdijzySBUf+WB96JLy2EJWctDOixSLxkoP6aU+tnBLJSltLYju6FoGKH7MIFshMe6IOQBshnGwryhIhr8LLRbX0iaOurv6r2QzpiS9tO+ar2VPs3xLKIi9ll8T2lfIovNQx5RBe6zuNlVrbxtoZ6Wr1A+mG9r10U+qtby6RfZMH2WSHpIt2Upa5aj5sqGaT0c6ivldpU4xDTsomDeXKhvmb+DFzgcrVPBCJdjm0v6JN9uVBDtLItmkD6alTY6Wm0z7IV7PrCO1V+ap/aB1D22bMbuANqzFmT8MiSCfVZbBIIF25cBLxBB0X7qB6aovbvkWQyqud8PvK7EP5avXFOwUlffVpsVbTTV99Wqjj0EMJeWrlRl2Xi1WI7YibLBZoLPhqqA24uIhV+JjFWkSLza5Fm/RD+TUk09hFn3RQ2iJoIRoXueiRsJp++vpe8tX6TxuXsv+k05pNAP1UywcaL7gSyVmTBdQXZTtoM20vbUly1uTo08lYO5M918rS5mQI0s2YuUTy0P7a5k3txEX9rJqPPujqd+WhTLFKm6TPWpwY20eME+QirDYP0MZldZbEeaq0PRHLrW0wkU/xNTutIZ0OkTPKiENnQ+dB5anZtTG7id9hNcYcGH70ox+lo97NKeGLws1iLPm/9rWvpeMQ+DhGM58O+pDGJql94fGlL31p9m2eWn3xoyDPetazsm9B11coI7X3qD7wgQ+kd7BA/QZ8RIT3r/TuWXTxi5aPPPJI9i141ate1fvFzRq8+9UsOJMf+6hx2WWXpSPvi636HlwN2WCtT7G5f/7zn7PHHnts3ib0SNiqP6NT67+un8X46le/mo6vec1r0rHkRS96UfY9k1o9wHt+fOQFzj777HQs4T3GGrSZtve9kzeGsXZ2wgknpCNx5XucvCvZZTsl68wl2HftIz1XX3119s1mv/rVr7JvwZh8vPvJeOB9zJpu1H+MBb1Duu782KW7sX1EG5Cr2bBV54FVbCe+q92V/+9//3v27ZwvBWHNRjb50V/fu7fiJz/5STpXLbMrbJF5hD5rNsMpDJ3x3nLt3eEuln1t2Jht4w2rMebAcOedd2ZfNyxmgJP6qrDg0IeRzOqwkIZ///vf6agPh7DwYgHc59gkbAItwPs47bTTsm82++Mf/5h967OODQILUj6wwuJ0SDuGEjfxXZvPVYibqbEXFkr0QaG4cRnKKnYWNx3Ueeqpp3Z+gGgMm5hL2DCKMRuPWr4//elP6Xj06NGqPqLr2sBtok2r9JEufK1rW+vQdUExfjjpD3/4Q/Z1w7i++OKL8191sL9bbrklnffoi5///OdzGx27aX3ggQeyz5jjgzesxhgT6LqjNAR9eZJN79/+9rfZL3/5yxxjVqG8E/6f//wnHdfdBGya2l2T4wkbAjZrbJr4ncYbbrhhfndlE6gfNs26d3GwCxbpbIQ+//nPz84999y0sRrLqnbGV1fvuOOOdOeLO3nauLI5G8tU5xL10Sp9tck2rdJHuvA1ReITBeimD30d+J3vfGcOeSboGvtD3/HCATYaN61f//rXk9+YqeMNqzHmwPHggw9mXzd6HHUo/GQDP1vChvfRRx9Nj7+t8oiZWaBF8XOe85x0FCy0hjw2t2l0V6ePIY9Aj+V3v/td9i2HOy9sCHg8Fj2xQB36OOpehr5h0c/vS37jG99Id5N0Z21VVrEz6uQxbTauzCFsLK644oq0kR7KNueSVe2zzDfkaZXIttp0vOaCMTz/+c/PvsUd6pJ41/dlL3tZ9tXR48B9F8r0Uzu1n7xhTtDd2XvuuScdl3HmmWdmnzHHB29YjTEHhjGP+45Z7LIY452uI0eOzK699trj+sjZfkKPob3tbW9Lx7jx6rszwN0F3CaIi8fae7ElXT/EvwqyV70vWoONmu7gceflkksuSYtZNq7bumCi9zXhZz/7WfatT9wUjXkXmA0LvzXJY8psVNfdoG/Czpg/2JhpYzD00eRtzyWvfvWrs28cynfOOeekI7ru66O4Qd9Gm1bpI1344vH4TT2lEW2264IW41BjechFr5e85CXZV2fI48DaxHc9sq/fhB3y+DFs40KcMWPwhtUYc2D48Ic/nI4stroWmrrifOmll6bjEHTyf9Ob3pSOexHdzdzku47rwIKSxRR3qeImUAs1dM7CrYR8H/rQh3a8V7oOvMenD3Fx566GbInHbje5SXzjG9+Yjtgrj/iW0NbLL798rh8eg4WLLrooHbcFd3akEz58s6nFf+xntWUId911V7qTif43tcEba2fYQLmBQ5YvfOEL+a9hm5V15hLu7NZQvV135cbkY6Oop0/e8573VNtEWLxzt635cWwf6cIXbOpR2HgBoO9ReZ17kLM2XobOIUMeBwZtzrvu6KoOzTE1Yt8uu+trzLbxhtUYc2DgroeudLOQKeGqNBs2FkJdd2n++te/Zt8z7wLpMSxRvrtG+TFP10JxW2gRwx27uGjibkh8n6xrA7LuO4Zd1OpjQcnC7Fvf+taOTchVV12VfbN0N5E+Racs+NA3j4WyYattXFbVtzZP2EbZ56B+X/a+5NNPP519w7jmmmvmmwM2hnwkpWwrC85ygVvekSV9vJOCvmsL/DHcdNNN6UgflR9vwa/H7ms679ID7YhfSy036YwflRfHoSCP7iwBfh4RjtTeJ62N6VXs7BOf+MQzbJl4Nnz045h3ncfOJcCFjdh+8elPfzoduy4CjM133XXXpSN9f/rpp6c7qOgFh5+73bw3XbJKm6AmG4ztI+J1oYU73uUYIJ944oknsq8f+lRjtC+Pzj3orLZZvv3221M5y+aQIY8Dg3Tzmc98pjq/qu3vfe9707FG3ICjS2OOK8eMMWaPEn+rD1f7vcoSfvOuOeGn9Oeff/78t/Moq1lQdP5GXfwtO35PkLz6DbxmkT2Pw89v1lFO/L1J6iSPyo6yd9VZI/5mXy1fs9mex8ffL4VYZ7M4SvJwJB0yKw5Z+RvQD2kULn2JmK/8nUVkQ0bFS19AOYqjXMlKHsqUXDUIV5mlo/2RWA9uiI3UoG3kRy6VIVkJ75JV+XBj+lnQ39J/6aI9QdQLccgWj4RTFnLotymjXWO7EcrWWCGf8oho96SjrTjKj3ZI3VE/Ma78vVXqjP2FH/mpi3JiHGHYFH2sMNkucdRT6j/K0TemYYydaWxRh9pEW1R/l32URJ3ipy0qU+G0EVnV96RRHGnVhmX1r5oPopylK8fY2Dbhyjxqa8mYPoJyPJEGeTjGOkmDDsr5robykb4P2kC7SRvnEOUfYiOSawjSDXVq7FIfdQ8pR/ZR06Mxu403rMaYPYcWh12OE20fnLRJo8U4jpN634IhLizIFxfaMQ7HCV4LHS384gITv9JGt0zuvnx9OonQRrVbi0aQPljMSE7CyrJwyAG1OByyLMsrkAd9KR4Z0OWyhSJ1lPlK/XXJgFuFsk4WfV2yLrNR4odC+dSjhTb9VrZVEB7TqR5tDtCTFq+SpXTkGdN/1ENc1Af5qYt42VOXTsoySa/8xKtvCSctLo4/4O+YXhsCZEEuXDm+KU/jljxlmYDMy+wMSIcekE1pceRVHwwhyqT8sq/aXALqK8Jpt/Sg/F31r5pPxL7H4a/pcEybqFPpStclD+GUqXRdfSRq40m2wd/ElRdn+iCtylmGdCFda8wMqU9jeIxspW6ob0jfgvLV+tSY3eYQ/zQGaYwxxhhj9hg80s8jrs2mL310aiir5jPPRL913GyGN/oe+/HkpJNOSr+FbdswU8DvsBpjjDHGGLMi/FTMiSeemD4Ath/gHVfet629i2zM8cAbVmOMMcYYY1aEu6psWm+55ZbOj0TtJdioHj16dO2fiDJmU3jDaowxxhizx1n1K9i7/bXy/QpfAj58+HD1J6j2EsjP18f5zVxjpoI3rMYYY4wxexB+skS/ecr7k/GnWfpYNZ/p5+abb55ddtlladNX+zmZqcN7zTzazN1iY6aEP7pkjDHGGLPH0EeTavQt7VbNZ4bz8MMPz773ve+lDexegU32pZde6seAzSTxhtUYY4wxxhhjzCTxI8HGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYyaJN6zGGGOMMcYYYybIbPb/Ab4GTtJlqIpxAAAAAElFTkSuQmCC\" style=\"width: 603px;\"\u003e\u003cbr\u003e\u003c/p\u003e\n\u003cp\u003eThe Discrimination index (R) was calculated as such:\u003c/p\u003e\n\u003cp\u003eR\u0026thinsp;=\u0026thinsp;H/n-L/n\u003c/p\u003e\n\u003cp\u003eWhere:\u003c/p\u003e\n\u003cp\u003e\u0026ldquo;H\u0026rdquo; is the number of top 27% candidates with a total score of 70% (28 out of 40) and above.\u003c/p\u003e\n\u003cp\u003e\u0026ldquo;L\u0026rdquo; is the number of bottom 27% candidates with a total score of 70% (28 out of 40) and above.\u003c/p\u003e\n\u003cp\u003e\u0026ldquo;n\u0026rdquo; is the total number of 27% of the candidates.\u003c/p\u003e\n\u003cp\u003eThe interpretation of difficulty and discrimination indices are given in supplemental file B.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eDescriptive Statistics was analysed with reliability (internal consistency) measures applied on all six stations by using Cronbach\u0026rsquo;s alpha (α). Pearson correlation (r) was performed between MDCAT scores, F.Sc. Scores and MMI scores to assess significant correlation. As shown in table 1 the mean scores on all items are ranged from 7.1\u0026ndash;7.9 and SD ranged from. The reliability (Cronbach\u0026rsquo;s alpha) of scores on each construct ranged from 0.92\u0026ndash;0.95. The overall reliability (Cronbach\u0026rsquo;s α) of scores on MMI stations was 0.87.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;2a represents weak positive correlation between the MDCAT and the total score of the MMI exam with a significant p-value which means students who scored well on MDCAT have also scored well in MMI exam and vice versa. The F.Sc. scores correlated negatively with MMI scores.\u003c/p\u003e \u003cp\u003eWith regards to correlation between the MMI scores of each station, all stations showed positive but weak correlation with significant p-value except Communication \u0026amp; Counseling and Leadership which correlated negatively (Table\u0026nbsp;2b).\u003c/p\u003e \u003cp\u003eThe competency assessment covered six distinct domains, each was assessed using key metrics like the Difficulty Index and Discrimination Index to gauge the performance and effectiveness of individual items (Ethics, Leadership, Critical thinking, Ethics, Motivation, Communication skills). The easiest item were from the station of Ethics (Difficulty index\u0026thinsp;=\u0026thinsp;0.77) while the most difficult item were from stations assessing Leadership (Difficulty index\u0026thinsp;=\u0026thinsp;0.57). The station of Critical thinking exhibited the highest discriminating index (0.78) while the domain of Motivation displayed the lowest discriminating index (0.53). All six stations had high discrimination indices\u0026thinsp;\u0026gt;\u0026thinsp;0.53. Figure\u0026nbsp;1 shows the comparison between Difficulty and Discrimination indices of each MMI station.\u003c/p\u003e "},{"header":"Discussion","content":"\u003cp\u003eThe primary focus of this study was to explore the reliability and quality of Online Multiple Mini-interviews by post hoc analysis. The results suggest that online MMI had good reliability and reasonable validity. The stations were able to discriminate between high achievers and low achievers.\u003c/p\u003e \u003cp\u003eThe overall reliability of MMI scores in this study was 0.87 while the reliability for scores on individual stations was calculated to be within 0.92 and 0.95. The reliability noted in this study is in the higher range as the literature reports the reliability of MMI between 0.54 and 0.98 (\u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e). The reliability of MMI scores conducted face-to-face at SMDC in a previous cohort was 0.7 (\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e). The authors believe that comparatively higher reliability reported in online MMI could be due to the familiarity of the faculty members with the process of MMI. Compared to previous cohorts, this time the faculty went through robust training sessions before MMI. Additionally, most of the faculty members had already been a part of MMI and hence were better prepared. Another change was that this time the scenarios had undergone a three-stage vetting process (via Medical educationists, faculty members, and intermediate-level students).\u003c/p\u003e \u003cp\u003e The significant positive correlation between scores obtained at different stations (except leadership with communication skills) confirmed the convergent validity of the interviews. Although the constructs of each station are distinct, they assess the soft (non-cognitive) skills of the candidate. This explains the consistent performance of the candidates across most of the stations. Knorr et al. explain the phenomenon by emphasizing that the non-cognitive skills assessed via MMI overlap conceptually even if they are not identical (\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e).\u003c/p\u003e \u003cp\u003eMDCAT and MMI scores show a weak but significant positive correlation while non-significant negative correlation was seen between scores obtained in F.Sc. and MMI. In Pakistan, there are numerous examining bodies for F.Sc. which results in the variability. Apart from different boards of examinations, Higher Education Commission of Pakistan also issues equivalence certificates to students who have taken O and A level or American high school exams. The students in this cohort had taken F.Sc. examinations from different boards, while some had submitted their equivalence certificates. However, all the candidates had taken the same MDCAT exam. Literature suggests that there is no correlation between MMI scores and academic measures like the past academic performance of a candidate (Grade point average-GPA) (\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e) (\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e). Academic performance of a candidate in the form of grade point average (GPA) or MDCAT score reflects the candidate\u0026rsquo;s performance on knowledge-based tests while MMI assesses the non-cognitive skills, hence it is expected that good performance in the knowledge-based assessment will not ensure a better performance in non-cognitive assessment. The positive correlation among the scores found in this study needs to be further explored.\u003c/p\u003e \u003cp\u003eThe difficulty level of MMI stations ranged from easy (0.57) to moderately difficult (0.77) which means that fifty-seven percent to seventy-seven percent students were able to respond satisfactorily to the questions asked at these stations.. It is quite possible that the candidates had prior knowledge of the constructs that would be assessed and hence may have undergone some sort of preparation. This indicates a need to review the scenarios and revise them or develop new ones.\u003c/p\u003e \u003cp\u003eThe discrimination index was found to be in the range of 0.53\u0026ndash;0.78, suggesting that the stations were good discriminators. This indicates that the scenarios given in MMI were able to discriminate well between the candidates and hence contributed effectively in evaluating them on the given constructs.\u003c/p\u003e \u003cp\u003eThe limitation of this study is that the results are based on the performance of a single cohort and cannot be generalized. Secondly, video recording of the interviews was not done and only one examiner was present during the interview on each station.\u003c/p\u003e \u003cp\u003eIt is recommended that MMI should be conducted to assess the non-cognitive attributes of students. Since online MMI use fewer resources and is more feasible it may be a preferred method for conducting MMI.\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eThe healthcare of the country depends on the competency of medical graduates so it is crucial to have robust admission criteria. The medical graduates in addition to possessing the requisite knowledge to be competent doctor must also possess the soft skills to deal with the patients and colleagues appropriately. Therefore, admission criteria for medical colleges should consist of an assessment of non-academic attributes in addition to their past assessment scores. The psychometric analysis of MMI has demonstrated adequate reliability and quality (validity, discrimination and difficulty index) and hence, MMI should continue to be used. There is a need to conduct longitudinal studies to assess the soft skills of the students throughout their medical college studies specially during clinical postings and also followed during their house job period.\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003eEthics approval and consent to participate: Ethical approval was granted by the Institutional Review Board (IRB) Shalamar Medical and Dental College (REF: SMDC-IRB/AL/44/2022) IRB number: 0484.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe, the authors, hereby give our consent for the publication of the manuscript titled. We affirm that all authors have contributed significantly to the study and agree to be accountable for all aspects of the work, ensuring its accuracy and integrity. Furthermore, we declare that this manuscript is not under consideration for publication elsewhere and has not been previously published in whole or in part.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request. Additionally, any materials necessary to replicate the experiments conducted in this study will be made available upon request to the corresponding author\u003c/p\u003e\n\u003ch2\u003eCompeting interests\u003c/h2\u003e\n\u003cp\u003eThe authors declare that they have no competing interests\u003c/p\u003e\n\u003ch2\u003eFunding\u003c/h2\u003e\n\u003cp\u003eThis research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors\u003c/p\u003e\n\u003ch2\u003eAuthors\u0026apos; contributions\u003c/h2\u003e\n\u003cp\u003eSundus Iftikhar: acquisition of data, interpretation of results, and manuscript writing (introduction, methodology, and discussion).\u003c/p\u003e\n\u003cp\u003eSyed Hasan Shoaib: Reviewing and writing the manuscript and acquisition of data.\u003c/p\u003e\n\u003cp\u003eShaur Sarfaraz: Analysis of the data, interpretation of the work. Writing of the manuscript (results and discussion). Editing of the manuscript.\u003c/p\u003e\n\u003cp\u003eSyeda Kauser: conception of the work, interpretation of the data, planning the project, editing of the manuscript.\u003c/p\u003e\n\u003ch2\u003eAcknowledgment:\u0026nbsp;\u003c/h2\u003e\n\u003cp\u003eThe authors gratefully acknowledge Professor Zahid Bashir for his leadership and guidance, and the IT department of Shalamar Medical and Dental College for their valuable contributions in running the system smoothly.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eThe 4 Types of Reliability in. Research | Definitions \u0026amp; Examples. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.scribbr.com/methodology/types-of-reliability/\u003c/span\u003e\u003cspan address=\"https://www.scribbr.com/methodology/types-of-reliability/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePakistan Medical Commission. Admission Regulations (Amended) 2020\u0026ndash;2021. Vol. 8. 2021. p. 2020\u0026ndash;1.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMedical MY. Multiple mini interview as an admission tool in higher education: insights from a systematic review. J Taibah Univ Med Sci [Internet]. 2019;14(3). \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jtumed.2019.03.006\u003c/span\u003e\u003cspan address=\"10.1016/j.jtumed.2019.03.006\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDore KL, Kreuger S, Ladhani M, Rolfson D, Kurtz D, Kulasegaram K, et al. The Reliability and Acceptability of the Multiple Mini-Interview as a Selection Instrument for Postgraduate Admissions. Acad Med. 2010;85(10 SUPPL):S60\u0026ndash;3. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0b013e3181ed442b\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0b013e3181ed442b\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training-A systematic review. Vol. 35, Med Teach. 2013. p. 1027\u0026ndash;41. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.3109/0142159X.2013.829912\u003c/span\u003e\u003cspan address=\"10.3109/0142159X.2013.829912\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://www.tandfonline.com/doi/full/10.3109/0142159X.2013.829912\u003c/span\u003e\u003cspan address=\"http://www.tandfonline.com/doi/full/10.3109/0142159X.2013.829912\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBurgess A, Roberts C, Sureshkumar P, Mossman K, BMC Med Educ. Multiple mini interview (MMI) for general practice training selection in Australia: Interviewers\u0026rsquo; motivation. Vol. 18,. BioMed Central Ltd.; 2018. p. 21. Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s12909-018-1128-z\u003c/span\u003e\u003cspan address=\"10.1186/s12909-018-1128-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNenad MW. The multiple mini-interview and dental hygiene admissions: A feasible option? J Dent Educ [Internet]. 2020 Jun 1 [cited 2021 Mar 30];84(6):634\u0026ndash;41. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttp://doi.wiley.com/10.1002/jdd.12114\u003c/span\u003e\u003cspan address=\"http://doi.wiley.com/10.1002/jdd.12114\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJerant A, Henderson MC, Griffin E, Hall TR, Kelly CJ, Peterson EM et al. Do Admissions Multiple Mini-Interview and Traditional Interview Scores Predict Subsequent Academic Performance? A Study of Five California Medical Schools. Acad Med. 2019;94(3):388\u0026ndash;95. DOI: 10.1097/ACM.0000000000002440 Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://journals.lww.com/academicmedicine/Fulltext/2019/03000/Do_Admissions_Multiple_Mini_Interview_and.31.aspx\u003c/span\u003e\u003cspan address=\"https://journals.lww.com/academicmedicine/Fulltext/2019/03000/Do_Admissions_Multiple_Mini_Interview_and.31.aspx\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKumar N, Bhardwaj S, Rahman E. Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom: a cohort study. Adv Med Educ Pract. 2018;9:239. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2147/AMEP.S159412\u003c/span\u003e\u003cspan address=\"10.2147/AMEP.S159412\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e Available from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e/pmc/articles/PMC5903841/\u003c/span\u003e\u003cspan address=\"http:///pmc/articles/PMC5903841/\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaider SI, Bari MF, Ijaz S. Using multiple mini-interviews for students\u0026rsquo; admissions in pakistan: A pilot study. Adv Med Educ Pract. 2020;11:179\u0026ndash;85. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2147/AMEP.S246285\u003c/span\u003e\u003cspan address=\"10.2147/AMEP.S246285\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUngtrakul T, Lamlertthon W, Boonchoo B, Auewarakul C. Virtual Multiple Mini-Interview during the COVID-19 Pandemic. Med Educ. 2020;54(8):764\u0026ndash;5. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1111/medu.14207\u003c/span\u003e\u003cspan address=\"10.1111/medu.14207\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eCarroll Turpin MA, Steele K, Matuk-Villazon O, Rowland K, Dayton CB, Horn KV. Rapid transition to a virtual multiple mini-interview admissions process: A new medical school\u0026rsquo;s experience during the COVID-19 pandemic. Acad Med. 2021;96(8):1152\u0026ndash;5. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.1097/ACM.0000000000004179\u003c/span\u003e\u003cspan address=\"10.1097/ACM.0000000000004179\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTiller D, O\u0026rsquo;Mara D, Rothnie I, Dunn S, Lee L, Roberts C. Internet-based multiple mini-interviews for candidate selection for graduate entry programmes. Med Educ. 2013;47(8):801\u0026ndash;10. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/medu.12224\u003c/span\u003e\u003cspan address=\"10.1111/medu.12224\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAziz I, Shoaib SH, Iftikhar S, Bashir MZ, Ashraf A, Shahzad M. Inducting undergraduate medical students via WhatsApp-based multiple mini interviews during COVID-19 pandemic. J Pak Med Assoc. 2022;72(11):2270\u0026ndash;4. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003e10.47391/JPMA.5325\u003c/span\u003e\u003cspan address=\"10.47391/JPMA.5325\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDela Pena R Jr, Musa L. Post-Hoc Analysis of the Departmental Final Examinations for Selected Gen., editors. Subjects for Arellano University, Ay 2010-11. 2011.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYusoff MSB. Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review. J Taibah Univ Med Sci. 2019;14(3):203\u0026ndash;40. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jtumed.2019.03.006\u003c/span\u003e\u003cspan address=\"10.1016/j.jtumed.2019.03.006\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKnorr M, Schwibbe A, Ehrhardt M, Lackamp J, Zimmermann S, Hampe W. Validity evidence for the Hamburg multiple mini-interview. BMC Med Educ. 2018;18(1):1\u0026ndash;12. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1186/s12909-018-1208-0\u003c/span\u003e\u003cspan address=\"10.1186/s12909-018-1208-0\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDuff RE, Katcher PA, Daniels RM, Ramaswamy V. The Multiple Mini Interview as a Dental School Admission Tool: Can It Predict Noncognitive Traits Associated with Professional Behaviours? J Dent Educ. 2020;84(4):478\u0026ndash;85. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21815/JDE.019.184\u003c/span\u003e\u003cspan address=\"10.21815/JDE.019.184\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAli S, Hashmi MSS, Umair M, Beg MA, Huda N. Multiple Mini-Interviews: Current Perspectives on Utility and Limitations. Adv Med Educ Pract. 2019;10:1031. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.2147/AMEP.S181332\u003c/span\u003e\u003cspan address=\"10.2147/AMEP.S181332\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTables 1 and 2 are available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Reliability of Results, college admission test, admission criteria, school","lastPublishedDoi":"10.21203/rs.3.rs-4110146/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4110146/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground:\u003c/strong\u003e MMI is a widely used method for assessing the non-cognitive skills of students, but the logistics and costs associated with organizing in-person MMI can be substantial. Virtual MMI, such as those conducted through platforms like WhatsApp Video calls, offer increased convenience, yet their reliability and quality remain uncertain. The objective of the study is to determine the reliability and quality (difficulty and discrimination indices) metrics of MMI scores conducted through WhatsApp Video call.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods: \u003c/strong\u003eSix MMI stations were used to assess the non-cognitive attributes of the students. In this descriptive study, the scores obtained by 678 students in MMI, F.Sc (equivalent of high school) and MDCAT were used to analyze the data. Item analysis was employed to assess quality of MMI stations. The reliability was calculated using Cronbach’s alpha and Pearson correlation (r) was performed between MDCAT scores, F.Sc. Scores and MMI scores to assess significant correlation.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults: \u003c/strong\u003eThe overall reliability of MMI in this study was 0.87 while the reliability for each interview station ranged between 0.92 - 0.95. The difficulty level of MMI stations ranged from easy (0.57) to moderately difficult (0.77). The discrimination index was found to be in the range of 0.53-0.78.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusion:\u003c/strong\u003e The psychometric analysis of MMI scores demonstrated good reliability and quality (discrimination and difficulty index) with the stations showing acceptable discrimination and moderate difficulty. Hence, online multiple mini-interviews were found useful to assess non-cognitive skills for admission in MBBS program.\u003c/p\u003e","manuscriptTitle":"Reliability and Quality of Online Multiple Mini interviews for Admissions in an MBBS program","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-03-25 06:20:24","doi":"10.21203/rs.3.rs-4110146/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-04-26T06:29:45+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-04-13T09:44:15+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-04-09T15:56:44+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"ee9e1450-14e6-4932-ad95-9c73b6d35ac8","date":"2024-03-31T06:01:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"2b1ea3b2-4f12-49a1-8281-3fdabdb763e9","date":"2024-03-30T10:33:20+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-03-30T08:18:15+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-03-24T07:59:42+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-03-21T07:56:55+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-03-21T07:53:17+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2024-03-15T20:03:42+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"1f747cdf-76f5-4daa-a061-94dee72288b0","owner":[],"postedDate":"March 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"under-review","subjectAreas":[],"tags":[],"updatedAt":"2025-05-05T16:23:18+00:00","versionOfRecord":[],"versionCreatedAt":"2024-03-25 06:20:24","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4110146","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4110146","identity":"rs-4110146","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}