Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions

Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research article Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions THOMAS PUTHIAPARAMPIL, Md Mizanur Rahman This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-16617/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 06 May, 2020 Read the published version in BMC Medical Education → Version 1 posted 10 You are reading this latest preprint version Abstract Background Multiple choice questions, used in medical school assessments for decades, have many drawbacks, such as: hard to construct, allow guessing, encourage test-wiseness, promote rote learning, provide no opportunity for examinees to express ideas, and do not provide information about strengths and weakness of candidates. Directly asked and answered questions like Very Short Answer Questions (VSAQ) is considered a better alternative with several advantages. Objectives This study aims to substantiate the superiority of VSAQ by actual tests and obtaining feedback from the stakeholders. Methods Conduct multiple true-false, one best answer and VSAQ tests in two batches of medical students, compare their scores and psychometric indexes of the tests and seek opinions from students and academics regarding these assessment methods. Results Multiple true-false and best answer test scores showed skewed results and low psychometric performance compared to better psychometrics and more balanced student performance in VSAQ tests. The stakeholders’ opinions were significantly in favour of VSAQ. Conclusion and recommendation This study concludes that VSAQ is a viable alternative to multiple choice question tests, and it is widely accepted by medical students and academics in the medical faculty. Educational Philosophy and Theory Internal Medicine Very Short Answer Questions Multiple Choice Questions Best answer questions Figures Figure 1 Figure 2 Figure 3 Introduction Multiple True/False (MTF) and One Best Answer Questions (BAQ) are widely employed in medical faculties by virtue of their advantages of instant machine scoring, freedom from examiner bias and dependable reliability [ 1 – 4 ]. In this article, ‘MCQ’ is used to refer to both these instruments of assessment. Reliability of a test is higher when the subject coverage is wider [ 5 , 6 ]. Reliability refers to test reproducibility with similar results when used for different batches, but it does not ensure validity [ 2 , 3 ]. MCQ tests seem to sacrifice validity for reliability [ 3 ]. A primary purpose of any assessment is to let students know what is important to learn [ 6 ]. Assessments are supposed also to enable feedback to students and direct the teaching strategies [ 1 , 7 – 10 ], and this is applicable especially to formative assessments. MCQ tests do not provide information, which would enable feedback, as they do not require students to construct the answers [ 1 ]. It is an established fact that assessments drive the learning style [ 1 , 6 , 10 , 11 ]. MCQ is blamed for promoting rote learning, guessing, test-wiseness, and turning students into data banks [ 2 , 5 , 7 , 12 – 14 ]. MCQ tend to test trivia [ 2 ] and they are not able to test complex issues [ 3 , 7 ]. MCQ does not provide any opportunity for students to express their understanding [ 15 ]. What is required in a medical school is learning that leads to the formation of competent doctors, which MCQ tests are incapable of assessing fully [ 3 ]. Constructing good MCQs with appropriate difficulty and discriminating ability needs expertise and experience, which are often lacking [ 6 , 15 , 16 ]. Discrimination index (DISi) in MCQ may be deceptive as good students may find them more difficult to answer, as the questions may be the authors’ opinions and not well-known facts [ 2 , 15 ]. True-false MCQ may even have a negative effect on students, attributable to the false statements, which they might take home as true [ 10 ]. It appears that the demerits of MCQ are numerous and overwhelming, and there is strong backing for alternative instruments like directly asked directly answered questions. Objectives of the study This study aims to test directly asked directly answered questions like very short answer questions (VSAQ) in medical students, compare their performance with MCQ tests and seek opinions from participating students and academics of the Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, regarding MCQ and VSAQ. Materials And Methods Thirty-seven year-3 and 39 year-5 medical students took part in the study. VSAQ papers were prepared by converting MTF and BAQ items used for end of posting examinations of the same batches of students but of different rotations of the posting. This meant that the questions were testing the same knowledge using different instruments in different students of the same standard. The VSAQ answer scripts were manually scored. The student scores of MTF, BAQ and VSAQ tests were compared using Pearson correlation coefficient. Student feedbacks were obtained using a written questionnaire about their views on MTF, BAQ and VSAQ. The item analysis of VSAQ, manually performed, was compared with those of MTF and BAQ. The academics’ views were obtained using a Google Form questionnaire with 17 items. Results Table 1 Year-3 and year-5 Students’ scores in the 6 Tests. Test Y3 Q Mean (SD) Min, Max Y5 Q Mean (SD) Min, Max §p-value MTF 37 20 45.5 (9.4) 25, 54 39 20 55.0 (11.1) 29, 76 p < 0.001*** BAQ 37 15 54.9(12.8) 31.7, 82.3 39 15 76.9 (11.6) 53.3, 100 p < 0.001*** VSAQ 37 53 54.1(12.4) 33.7, 77.9 39 32 62.9 (11.2) 38, 91 p < 0.01** VSAQ vs. MTF (r = 0.714***) VSAQ vs. MTF (r = 0.632***) VSAQ vs. BAQ (r = 0.435**) VSAQ vs. BAQ (r = 0.321*) *p < 0.05, **p < 0.01, ***p < 0.001. §p-value reached from independent sample t test Y3 = year-3, Y5 = year-5 number of students, Q = number of questions, Min = minimum, Max = maximum, SD = standard deviation, MTF = multiple true-false test, BAQ = one best answer test, VSAQ = very short answer questions test. Table 1 shows significantly higher performance of year-5 students in MTF, BAQ and VSAQ compared to year 3 (p < 0.01). Test-wise analysis reveals a strong correlation between MTF and VSAQ both in year-3 and year-5 tests (p < 0.001). However, a weak positive relationship is found between BAQ and VSAQ (p < 0.01). The scores of 2 batches of students (year-3, 37 and year-5, 39) in each of 3 tests were combined for this chart. The scores (%) achieved are shown in 4 categories (< 50, 50–59, 60–74 and ≥ 75). For VSAQ preparation the previous rotation of same batch students’ question papers was used. The overall performance in MTF shows a low trend in contrast to BAQ, which shows an upward trend. However, the performance in VASQ shows a desirable, balanced distribution (Fig. 1 ). The item analysis values of 3 tests of year-3 and 3 tests of year-5 batches were combined for this figure. The bars show percentages of items falling in 3 categories: ≤0.3 (too difficult), 0.31–0.79 (moderate difficulty) and ≥ 0.8 (too easy) as per Ebel and Frisbe (1972). Most MTF items fall in the moderate difficulty category, BAQ shows a trend towards more easy items, while VSAQ shows a more balanced distribution. The discrimination index of year-3 and year-5 test items are shown in Fig. 3 . MTF (20 items), BAQ (15 items, 4-option) and VSAQ (54 items Y-3 and 32 items Y-5) are combined in this figure. Different groups of students of the same batch participated in the study. The VSAQ papers were prepared by converting the MTF and BAQ questions. The bars show percentages of items falling in the three categories: ≤0.0 = no or negative discrimination; 0.1–0.19 = fair and ≥ 0.2 good and excellent distribution combined. MTF shows a fair distribution, BAQ higher than MTF and VSAQ about 70% of items having the highest discrimination power. Item analysis of VSAQ was manually performed on Excel Worksheet using the UNIMAS formulae [ 17 ]. Table 2 Students’ feedback on VSAQ Statements Year-3 (n = 37) Year-5 (n = 39) p-value In favour Neutral Against In favour Neutral Against VSAQ is an efficient method of assessment of knowledge 86.5 13.5 0.0 84.6 10.3 0.0 p > 0.05 VSAQ can test more items compared to MTF and BAQ in the same amount of time 70.3 10.8 18.9 59.0 33.3 17.9 p 0.05 VSAQ provides an opportunity for students to express their ideas 86.5 10.8 2.7 89.7 7.7 2.6 p > 0.05 Happy to see a new assessment method 78.4 21.6 0.0 84.6 12.8 0.0 p > 0.05 I would like to see more VSAQ tests. 59.5 35.1 5.4 79.5 15.4 5.1 p > 0.05 *p < 0.05, **p < 0.01, ***p < 0.001. p-value reached from chi-square test Most of the students expressed opinions in favour of VSAQ. More than four-fifths (86.5%) of them opined that VSAQ is an efficient method of assessment of knowledge, and equal proportions of them opined that VSAQ provides an opportunity for students to express their ideas. About 80% in both groups were happy to see this new method of assessment, and 59.5% and 79.5% of them expressed interest to see more VSAQ tests (Table 2 ). Pearson’s chi-square test of independence revealed no statistically significant difference between year-3 and year-5 students’ opinions (p > 0.05). The results showed high acceptance of VSAQ compared to MCQ. Table 3 Students’ feedback on MTF and BAQ Statements Year-3 (n = 37) Year-5 (n = 39) p-value Agree Not sure Disagree Agree Not sure Disagree MTF is more efficient than VSAQ for testing knowledge 16.2 64.9 18.9 15.4 46.2 38.5 p > 0.05 BAQ is more efficient than VSAQ in testing knowledge 29.7 40.5 29.7 15.4 41.0 43.6 p > 0.05 MTFs, which I encountered had been clear and unambiguous 29.7 29.7 40.5 28.2 15.4 56.4 p > 0.05 BAQs are easy, as most distractors are easy to exclude to reach the answer 62.2 10.8 27.0 53.8 20.5 25.6 p > 0.05 Very often Sometimes Seldom Very often Sometimes Seldom Did you encounter confusing items in MTF? 0.0 10.8 89.2 59.0 35.9 5.1 p < 0.001 Did you encounter confusing items in BAQ? 0.0 43.2 56.8 28.2 51.3 20.5 p < 0.001 Very confident OK Not confident Very confident OK Not confident How confident have you been while answering MTF? 0.0 73.0 27.0 53.8 20.5 25.6 p < 0.001 *p < 0.05, **p < 0.01, ***p < 0.001. p-value reached from chi-square test There is a significant similarity in the opinions expressed by year-3 and year-5 students about MCQ tests. They did not consider MCQ tests any better in assessing knowledge compared to VSAQ. Over 62% (year-3) and 53% (year-5) students considered BAQ easy to answer, as they contained many distractors easy to exclude as the answer. A good number of students, more of them in year-5 group, had encountered confusing MTF and BAQ items in their examinations. Lecturers’ feedback on MTF, BAQ and VSAQ More than 90% were involved in constructing MCQ and BAQ, but less than 10% of them mentioned it was a pleasant experience. More than 90% felt writing BAQ was more difficult than writing MTF. Over 67% considered writing 3-4 plausible distractors a difficult task. In MTF, 87% considered writing of false options more difficult than writing true options. Twenty per cent of them said they converted true options to false, and some said they just wrote false options without care, as they were false anyway. About using item analysis to improve the questions, 60% mentioned they did not do it. Cent per cent respondents said they got ideas about the students’ strengths and weaknesses while marking directly asked-directly answered questions like MEQ and SAQ, and 40% of them said they do not get any such information from MTF and BAQ results. However, 97% believed such information was important to guide teaching. About their openness to VSAQ, 60% answered ‘surely’ and 40% answered ‘maybe’. Discussion Students scored poorly in MTF with high failure rates and very few high scorers. On the contrary, BAQ was skewed towards high scores with very few poor performers. The VSAQ scores showed a fair and balanced distribution as would be expected in a reliable test. There was a strong positive correlation between VSAQ and MTF both in year-3 and year-5 tests (p < 0.001), while the correlation between VSAQ and BAQ in both tests were weakly positive (p < 0.01). This discrepancy was somewhat reflected in the item analysis of these tests. While most MTF items fell in the moderate difficulty category, BAQ items were more easy and VSAQ more balanced in distribution. Items with 0.2 or higher DISi were more in VSAQ compared to MTF and BAQ, while items with lower DISi were more in both MTF and BAQ tests. Our overall results showed fairer and balanced student performance and superior psychometric properties of VSAQ compared to MTF and BAQ. Students’ opinions about MCQ were generally unfavourable. Mention of many confusing items in the questions was notable. Negative opinions about MTF and BAQ were more evident in year-5 students’ feedback, which reflected their seniority and the number of tests they would have taken compared to year-3 students. Generally, students’ opinions were highly in favour of VSQ. Hift [ 18 ] proposed elimination of open-ended questions like MEQ in summative assessments, as context-rich well written best answer questions had higher reliability and validity than MEQ. But this author also supported open-ended questions for formative assessments or assessments for learning. Open-ended questions were favoured also by college students [ 1 ]. Validity and reliability of tests depend on how widely the subject is covered in the assessment [ 5 , 6 ], but unduly prolonged tests would be counterproductive. Our VSAQ test with 53 questions and 14-item feedback was completed in 30–45 minutes. MTF of 20 questions took 50 minutes and BAQ of 15 questions took 45 minutes to complete. Our study showed that wider coverage of topics was easier to achieve in VSAQ tests compared to MCQ, as claimed in literature[ 3 , 19 ]. Test-wiseness and guessing, which are unavoidable in MCQ tests are of no concern in VSAQ. It has also been observed that students found false options harder than true ones to answer, and omission rates due to uncertainty are high in MTF [ 20 ]. The adverse effect of using false options was also highlighted by Wood [ 10 ]. BAQ claims higher-order assessment than MTF but can be too easy because of the many non-functioning distractors most of them contain [ 21 ]. Assessments are of immense value in letting students know what is important to learn [ 6 ] and for providing feedback to students and directing the teaching [ 1 , 7 , 8 , 15 , 22 ]. VSAQ was seen to achieve these ends better than MCQ. Reading the VSAQ answers while marking provides the examiner's insight into students’ strengths and deficiencies, which would help in providing feedback and modifying teaching [ 7 , 19 ]. MCQ tests do not reveal with certainty what the students know, how much they know, and whether they are capable of using the knowledge in real-life situations, as they do not require the students to construct the answers [ 1 ]. Directly answered questions like VSAQ is an alternative, which will reveal more of students’ competence. Factual knowledge, important for a doctor [ 15 ] was effectively tested by our VSAQ tests. Higher taxonomy testing is also said to be better achieved in VSAQ [ 1 ]. VSAQs are easier to construct [ 12 ]. The struggle of writing MCQ was reflected in our academics’ feedback. Most of them considered writing good MTF and 3 to 4 plausible distractors in BAQ a difficult task, just as described in the literature [ 21 ]. Our students’ feedback highlighted the issue of confusing items in MCQ, while no such opinions were expressed about VSAQ. The requirement to construct the answers would drive better learning, as preparation strategies would change [ 1 , 3 , 10 ]. The motivation for learning will be better when students are required to express their knowledge in the examination [ 1 ]. Direct questions let students express their ideas [ 1 , 19 ], which is a desired purpose of assessments. MCQ can be considered an assessment of convenience far from real-life medical practice, while direct questions are considered more natural [ 2 ]. In the past, when essay questions were used, the onus of constructing the answers was on students. With the advent of MCQ, the roles reversed. Now, the teachers need to cover the syllabus in questions, while the students do not need to write a single word, but to choose the correct answers either knowingly, by guessing or using test-wiseness. In effect, a switch from MCQ to VSAQ will transfer the burden of writing all the answers (true, false and distractors) from the teachers back to the students. Constructing questions will still be an expert job, albeit less arduous than MCQ, as it won’t require to create false options and plausible distractors. The only disadvantage envisaged in adopting VSAQ is the time spent on manual scoring. The time spent on constructing MCQ can now be utilised for marking scripts with more rewarding benefits. VSAQ can be scored by anyone with the help of model answers [ 7 , 12 ], and the possibility of computer technology [ 3 ] making the task easier in future is real. Our academics and students responded positively to a shift from MCQ to VSAQ. The demerits of MCQ are many, and there is strong backing for VSAQ, but the transition, as usual, is slow to come. Limitation of the study This study was limited to one institution and the student scores and comparisons were limited to one batch of students each of year-3 and year-5. Conclusion VSAQ employ directly asked questions, which require students to answer briefly and directly with no scope of guessing or reaching the answer by elimination. A larger quantity of knowledge recall, understanding and application can be tested in a shorter period compared to MCQ. The information obtained by the teachers while scoring will help to modify the teaching and to give feedback to the students. Previously used MTF and BAQ items can be converted to VSAQ effectively. Our study showed fairer student performance and better psychometric indexes in VSAQ compared to MTF and BAQ. The students by and large preferred VSAQ to MTF and BAQ. Our academics also expressed the deficiencies of MCQ and showed openness to VSAQ. Last but not the least VSAQ using computers will cut the costs and the carbon footprint of the faculty drastically. In conclusion, VSAQ will be a viable alternative to MCQ with many plus points. Abbreviations BAQ Best answer questions DIFi Difficulty index DISi Discrimination index IRB Institutional Review Board MCQ Multiple Choice Questions MTF Multiple true-false questions VASQ Very Short Answer Questions UNIMAS Universiti Malaysia Sarawak Declarations Acknowledgements Our thanks to the dean and deputy deans of the Faculty of Medicine and Health Sciences, UNIMAS for the approval of the project and permission to use the examination questions, results and item analysis reports for this project. Ethics approval The Technical Review Committee approved the study proposal of the Faculty of Medicine and Health Sciences, Universiti Malaysia of Sarawak (UNIMAS). Ethical clearance was also obtained from the Institutional Review Board (IRB) of the Faculty. Strict anonymity and confidentiality of the data were maintained. No student particulars were extracted and disclosed publicly. Students’ participation in this study was voluntary. The participants signed a formal consent form as per the faculty’s ethics committee requirements. The scores of these tests were not used for any formal assessment. Consent to publish Both authors read the manuscript and consented for publication The data and materials The data will not be available publicly. The principal author keeps all the materials and data used in this study. However, the printed questionnaire for this could be available with permission from the first author. Conflicts of interest There is no conflict of interest involved in this project. Funding Self-funded Authors’ contributions TP conceptualised the study, captured data and wrote the manuscript, MMR did the statistics, proofreading and editing of the manuscript. References Cirn JT. True/False versus Short Answer Questions. Coll Teach. 1986;34:34–7. https://www.jstor.org/stable/27558154. Accessed 10 Jan 2020. Pickering SG. Against Multiple Choice Questions. Med Teach. 1979;1:84–6. doi:10.3109/01421597909019397. Sam AH, Hameed S, Harris J, Meeran K. Validity of very short answer versus single best answer questions for undergraduate assessment. BMC Med Educ. 2016;16:266. van der Vleuten CPM, Schuwirth LWT. Assessing professional competence: from methods to programmes. Med Educ. 2005;39:309–17. Burton RF. Multiple‐choice and true/false tests: myths and misapprehensions. Assess Eval High Educ. 2005;30:65–72. doi:10.1080/0260293042003243904. Case SM, Swanson DB. Constructing Written Test Questions For the Basic and Clinical Sciences. Natl Board Med Exam. 2001;:181. http://www.medbev.umontreal.ca/docimo/DocSource/NBME_MCQ.pdf. Hauer KE, Boscardin C, Brenner JM, van Schaik SM, Papp KK. Twelve tips for assessing medical knowledge with open-ended questions: Designing constructed response examinations in medical education. Med Teach. 2019;:1–6. Martinez ME, Lipson JI. Assessment for Learning. Educ Leadersh. 1989;46:73–5. Van Der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. Twelve Tips for programmatic assessment. Med Teach. 2015;37:641–6. Wood T. Assessment not only drives learning, it may also help learning. Med Educ. 2009;43:5–6. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet Lond Engl. 2001;357:945–9. Chan CKY. Assessment: Short answer question, assessment resources,. 2009. https://ar.cetl.hku.hk/am_saq.htm. Accessed 12 Jan 2020. McCoubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004;26:709–12. Sim S-M, Rasiah RI. Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singapore. 2006;35:67–71. Anderson J. For Multiple Choice Questions. Med Teach. 1979;1:37–42. doi:10.3109/01421597909010580. Dutke S, Barenberg J. Easy and Informative: Using Confidence-Weighted True–False Items for Knowledge Tests in Psychology Courses. Psychol Learn Teach. 2015;14:250–9. doi:10.1177/1475725715605627. Puthiaparampil T. Assessment Analysis: How It Is Done. MedEdPublish. 2017;6. doi:10.15694/mep.2017.000142. Hift RJ. Should essays and other “open-ended”-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14:249. doi:10.1186/s12909-014-0249-2. Stone HL. Return to “basics” in medical education: a commentary. Med Teach. 1982;4:102–3. Puthiaparampil T, Gudum HR, Rahman MM, Saimon R, Lim IF. True-false analysis reveals inherent flaws in multiple true-false tests. Int J Community Med Public Health. 2019;6:4204–8. doi:10.18203/2394-6040.ijcmph20194476. Tarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 2009;9:40. doi:10.1186/1472-6920-9-40. Case SM, Swanson DB. Constructing Written Test Questions For the Basic and Clinical Sciences. 1998;:181. Cite Share Download PDF Status: Published Journal Publication published 06 May, 2020 Read the published version in BMC Medical Education → Version 1 posted Review # 2 received at journal 07 Apr, 2020 Editorial decision: Major revision 07 Apr, 2020 Reviewer # 2 agreed at journal 24 Mar, 2020 Review # 1 received at journal 24 Mar, 2020 Editor assigned by journal 12 Mar, 2020 Reviewers invited by journal 12 Mar, 2020 Reviewer # 1 agreed at journal 12 Mar, 2020 Submission checks completed at journal 05 Mar, 2020 Editor invited by journal 05 Mar, 2020 First submitted to journal 24 Feb, 2020 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-16617","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research article","associatedPublications":[],"authors":[{"id":395048,"identity":"69af5132-f73d-4a64-825a-205e700ecd64","order_by":1,"name":"THOMAS PUTHIAPARAMPIL","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAvUlEQVRIiWNgGAWjYFACHjApx8DA2ECkDjaIFmPStSQSq56BwVy+9+CjGzV16RtuN7du+PGHIdrgAAEtlm18ycY5xw7nbrhzsO1mbxtD7gZCWgyO8ZhJ57AdyN1wI7HtBm8D0Vr+1aUbALXc/POHWC25bcwJIC23ediI0GLZlmNsnNt32HAmSItsm0TuTEJazJnPGD7O+VYnz3cj/dnNN39scvsIOgyNL8GgQKoWBgb5BgJaRsEoGAWjYMQBAKdcSHohTHhLAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0002-7939-5185","institution":"Universiti Malaysia Sarawak","correspondingAuthor":true,"prefix":"","firstName":"THOMAS","middleName":"","lastName":"PUTHIAPARAMPIL","suffix":""},{"id":395049,"identity":"68f42ff3-4603-4d03-8339-bc25597f263a","order_by":2,"name":"Md Mizanur Rahman","email":"","orcid":"","institution":"Universiti Malaysia Sarawak","correspondingAuthor":false,"prefix":"","firstName":"Md","middleName":"Mizanur","lastName":"Rahman","suffix":""}],"badges":[],"createdAt":"2020-03-05 14:18:33","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-16617/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-16617/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12909-020-02057-w","type":"published","date":"2020-05-06T21:00:02+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":623774,"identity":"69d371a8-7131-4d26-9de4-5a8fb2feaad0","added_by":"auto","created_at":"2020-03-10 21:16:29","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":24438,"visible":true,"origin":"","legend":"Overall Student performance in 6 tests compared. p-value reached from Person’s Chi-square test of independence \nMTF vs. VSAQ (p\u003c0.05) and BAQ vs. VSAQ (p\u003c0.05)","description":"","filename":"Fiugre1.PNG","url":"https://assets-eu.researchsquare.com/files/rs-16617/v1/Fiugre 1.PNG"},{"id":623775,"identity":"868e4da8-f2d8-46aa-83bd-a8d112f218c7","added_by":"auto","created_at":"2020-03-10 21:16:29","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":13978,"visible":true,"origin":"","legend":"Percentage distribution of difficulty index of 6 tests .","description":"","filename":"Figure2.PNG","url":"https://assets-eu.researchsquare.com/files/rs-16617/v1/Figure 2.PNG"},{"id":623776,"identity":"e03df6ba-72c8-4baa-92ca-eaa0d1f7ce9c","added_by":"auto","created_at":"2020-03-10 21:16:29","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":15862,"visible":true,"origin":"","legend":"Percentage distribution of discrimination index of 6 tests","description":"","filename":"Figure3.PNG","url":"https://assets-eu.researchsquare.com/files/rs-16617/v1/Figure 3.PNG"},{"id":13493086,"identity":"2dc5dca4-162a-40e1-866d-7701fccf327f","added_by":"auto","created_at":"2021-09-16 22:34:14","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":330713,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-16617/v1/8a21f336-d4d1-4d85-9857-92354574d183.pdf"}],"financialInterests":"","formattedTitle":"Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions","fulltext":[{"header":"Introduction","content":" \u003cp\u003eMultiple True/False (MTF) and One Best Answer Questions (BAQ) are widely employed in medical faculties by virtue of their advantages of instant machine scoring, freedom from examiner bias and dependable reliability [\u003cspan additionalcitationids=\"CR2 CR3\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e4\u003c/span\u003e]. In this article, \u0026lsquo;MCQ\u0026rsquo; is used to refer to both these instruments of assessment. Reliability of a test is higher when the subject coverage is wider [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Reliability refers to test reproducibility with similar results when used for different batches, but it does not ensure validity [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. MCQ tests seem to sacrifice validity for reliability [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. A primary purpose of any assessment is to let students know what is important to learn [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e]. Assessments are supposed also to enable feedback to students and direct the teaching strategies [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan additionalcitationids=\"CR8 CR9\" citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e], and this is applicable especially to formative assessments. MCQ tests do not provide information, which would enable feedback, as they do not require students to construct the answers [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. It is an established fact that assessments drive the learning style [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e11\u003c/span\u003e]. MCQ is blamed for promoting rote learning, guessing, test-wiseness, and turning students into data banks [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan additionalcitationids=\"CR13\" citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e]. MCQ tend to test trivia [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e] and they are not able to test complex issues [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e]. MCQ does not provide any opportunity for students to express their understanding [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. What is required in a medical school is learning that leads to the formation of competent doctors, which MCQ tests are incapable of assessing fully [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e]. Constructing good MCQs with appropriate difficulty and discriminating ability needs expertise and experience, which are often lacking [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e16\u003c/span\u003e]. Discrimination index (DISi) in MCQ may be deceptive as good students may find them more difficult to answer, as the questions may be the authors\u0026rsquo; opinions and not well-known facts [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e]. True-false MCQ may even have a negative effect on students, attributable to the false statements, which they might take home as true [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. It appears that the demerits of MCQ are numerous and overwhelming, and there is strong backing for alternative instruments like directly asked directly answered questions.\u003c/p\u003e \u003cdiv id=\"Sec2\" class=\"Section2\"\u003e \u003ch2\u003eObjectives of the study\u003c/h2\u003e \u003cp\u003eThis study aims to test directly asked directly answered questions like very short answer questions (VSAQ) in medical students, compare their performance with MCQ tests and seek opinions from participating students and academics of the Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, regarding MCQ and VSAQ.\u003c/p\u003e \u003c/div\u003e "},{"header":"Materials And Methods","content":" \u003cp\u003eThirty-seven year-3 and 39 year-5 medical students took part in the study. VSAQ papers were prepared by converting MTF and BAQ items used for end of posting examinations of the same batches of students but of different rotations of the posting. This meant that the questions were testing the same knowledge using different instruments in different students of the same standard. The VSAQ answer scripts were manually scored. The student scores of MTF, BAQ and VSAQ tests were compared using Pearson correlation coefficient. Student feedbacks were obtained using a written questionnaire about their views on MTF, BAQ and VSAQ. The item analysis of VSAQ, manually performed, was compared with those of MTF and BAQ. The academics\u0026rsquo; views were obtained using a Google Form questionnaire with 17 items.\u003c/p\u003e "},{"header":"Results","content":" \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab1\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cdiv class=\"SimplePara\"\u003eYear-3 and year-5 Students\u0026rsquo; scores in the 6 Tests. \u003c/div\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"10\"\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eTest\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003eY3\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003eQ\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003eMean (SD)\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003eMin, Max\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003eY5\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003eQ\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003eMean (SD)\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c9\"\u003e \u003cdiv class=\"SimplePara\"\u003eMin, Max\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c10\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u0026sect;p-value\u003c/div\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eMTF\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e37\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e20\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e45.5 (9.4)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e25, 54\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e39\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e20\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e55.0 (11.1)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cdiv class=\"SimplePara\"\u003e29, 76\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.001***\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eBAQ\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e37\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e15\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e54.9(12.8)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e31.7, 82.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e39\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e15\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e76.9 (11.6)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cdiv class=\"SimplePara\"\u003e53.3, 100\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.001***\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e37\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e53\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e54.1(12.4)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e33.7, 77.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e39\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e32\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e62.9 (11.2)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c9\"\u003e \u003cdiv class=\"SimplePara\"\u003e38, 91\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.01**\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c5\" namest=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ vs. MTF (r\u0026thinsp;=\u0026thinsp;0.714***)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c9\" namest=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ vs. MTF (r\u0026thinsp;=\u0026thinsp;0.632***)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colspan=\"5\" nameend=\"c5\" namest=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ vs. BAQ (r\u0026thinsp;=\u0026thinsp;0.435**)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colspan=\"4\" nameend=\"c9\" namest=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ vs. BAQ (r\u0026thinsp;=\u0026thinsp;0.321*)\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c10\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"10\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003e*p\u0026thinsp;\u0026lt;\u0026thinsp;0.05, **p\u0026thinsp;\u0026lt;\u0026thinsp;0.01, ***p\u0026thinsp;\u0026lt;\u0026thinsp;0.001. §p-value reached from independent sample t test \nY3 = year-3, Y5 = year-5 number of students, Q = number of questions, Min = minimum, Max = maximum, SD = standard deviation, MTF = multiple true-false test, BAQ = one best answer test, VSAQ = very short answer questions test. \u003c/span\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab1\" class=\"InternalRef\"\u003e1\u003c/span\u003e shows significantly higher performance of year-5 students in MTF, BAQ and VSAQ compared to year 3 (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01). Test-wise analysis reveals a strong correlation between MTF and VSAQ both in year-3 and year-5 tests (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001). However, a weak positive relationship is found between BAQ and VSAQ (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01).\u003c/p\u003e \u003cp\u003eThe scores of 2 batches of students (year-3, 37 and year-5, 39) in each of 3 tests were combined for this chart. The scores (%) achieved are shown in 4 categories (\u0026lt;\u0026thinsp;50, 50\u0026ndash;59, 60\u0026ndash;74 and \u0026ge;\u0026thinsp;75). For VSAQ preparation the previous rotation of same batch students\u0026rsquo; question papers was used. The overall performance in MTF shows a low trend in contrast to BAQ, which shows an upward trend. However, the performance in VASQ shows a desirable, balanced distribution (Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e).\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe item analysis values of 3 tests of year-3 and 3 tests of year-5 batches were combined for this figure. The bars show percentages of items falling in 3 categories: \u0026le;0.3 (too difficult), 0.31\u0026ndash;0.79 (moderate difficulty) and \u0026ge;\u0026thinsp;0.8 (too easy) as per Ebel and Frisbe (1972). Most MTF items fall in the moderate difficulty category, BAQ shows a trend towards more easy items, while VSAQ shows a more balanced distribution.\u003c/p\u003e \u003cp\u003e \u003c/p\u003e \u003cp\u003eThe discrimination index of year-3 and year-5 test items are shown in Fig.\u0026nbsp;\u003cspan refid=\"Fig3\" class=\"InternalRef\"\u003e3\u003c/span\u003e. MTF (20 items), BAQ (15 items, 4-option) and VSAQ (54 items Y-3 and 32 items Y-5) are combined in this figure. Different groups of students of the same batch participated in the study. The VSAQ papers were prepared by converting the MTF and BAQ questions. The bars show percentages of items falling in the three categories: \u0026le;0.0\u0026thinsp;=\u0026thinsp;no or negative discrimination; 0.1\u0026ndash;0.19\u0026thinsp;=\u0026thinsp;fair and \u0026ge;\u0026thinsp;0.2 good and excellent distribution combined. MTF shows a fair distribution, BAQ higher than MTF and VSAQ about 70% of items having the highest discrimination power. Item analysis of VSAQ was manually performed on Excel Worksheet using the UNIMAS formulae [\u003cspan citationid=\"CR17\" class=\"CitationRef\"\u003e17\u003c/span\u003e].\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab2\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 2\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cdiv class=\"SimplePara\"\u003eStudents\u0026rsquo; feedback on VSAQ\u003c/div\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eStatements\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eYear-3 (n\u0026thinsp;=\u0026thinsp;37)\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eYear-5 (n\u0026thinsp;=\u0026thinsp;39)\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003ep-value\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eIn favour\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNeutral\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eAgainst\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eIn favour\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNeutral\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eAgainst\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ is an efficient method of assessment of knowledge\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e86.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e13.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e84.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e10.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ can test more items compared to MTF and BAQ in the same amount of time\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e70.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e10.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e18.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e59.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e33.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e17.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ tests knowledge better as the students need to write the answers\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e67.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e24.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e13.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e87.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e7.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e12.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eVSAQ provides an opportunity for students to express their ideas\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e86.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e10.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e2.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e89.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e7.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e2.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eHappy to see a new assessment method\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e78.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e21.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e84.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e12.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eI would like to see more VSAQ tests.\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e59.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e35.1\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e5.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e79.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e15.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e5.1\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003e*p\u0026thinsp;\u0026lt;\u0026thinsp;0.05, **p\u0026thinsp;\u0026lt;\u0026thinsp;0.01, ***p\u0026thinsp;\u0026lt;\u0026thinsp;0.001. p-value reached from chi-square test\u003c/span\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e\u003cp\u003eMost of the students expressed opinions in favour of VSAQ. More than four-fifths (86.5%) of them opined that VSAQ is an efficient method of assessment of knowledge, and equal proportions of them opined that VSAQ provides an opportunity for students to express their ideas. About 80% in both groups were happy to see this new method of assessment, and 59.5% and 79.5% of them expressed interest to see more VSAQ tests (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Pearson\u0026rsquo;s chi-square test of independence revealed no statistically significant difference between year-3 and year-5 students\u0026rsquo; opinions (p\u0026thinsp;\u0026gt;\u0026thinsp;0.05). The results showed high acceptance of VSAQ compared to MCQ.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cdiv class=\"SimplePara\"\u003eStudents\u0026rsquo; feedback on MTF and BAQ\u003c/div\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"8\"\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\" morerows=\"1\" rowspan=\"2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eStatements\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c4\" namest=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eYear-3 (n\u0026thinsp;=\u0026thinsp;37)\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colspan=\"3\" nameend=\"c7\" namest=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003eYear-5 (n\u0026thinsp;=\u0026thinsp;39)\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c8\" morerows=\"1\" rowspan=\"2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003ep-value\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003c/tr\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eAgree\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNot sure\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eDisagree\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eAgree\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNot sure\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eDisagree\u003c/span\u003e\u003c/div\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eMTF is more efficient than VSAQ for testing knowledge\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e16.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e64.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e18.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e15.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e46.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e38.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eBAQ is more efficient than VSAQ in testing knowledge\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e29.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e40.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e29.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e15.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e41.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e43.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eMTFs, which I encountered had been clear and unambiguous\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e29.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e29.7\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e40.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e28.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e15.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e56.4\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eBAQs are easy, as most distractors are easy to exclude to reach the answer\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e62.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e10.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e27.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e53.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e20.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e25.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003ep\u0026thinsp;\u0026gt;\u0026thinsp;0.05\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eVery often\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eSometimes\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eSeldom\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eVery often\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eSometimes\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eSeldom\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eDid you encounter confusing items in MTF?\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e10.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e89.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e59.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e35.9\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e5.1\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eDid you encounter confusing items in BAQ?\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e43.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e56.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e28.2\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e51.3\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e20.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e\u0026nbsp;\u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eVery confident\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eOK\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNot confident\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eVery confident\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eOK\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003eNot confident\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e\u0026nbsp;\u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cdiv class=\"SimplePara\"\u003eHow confident have you been while answering MTF?\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c2\"\u003e \u003cdiv class=\"SimplePara\"\u003e0.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c3\"\u003e \u003cdiv class=\"SimplePara\"\u003e73.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c4\"\u003e \u003cdiv class=\"SimplePara\"\u003e27.0\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c5\"\u003e \u003cdiv class=\"SimplePara\"\u003e53.8\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c6\"\u003e \u003cdiv class=\"SimplePara\"\u003e20.5\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c7\"\u003e \u003cdiv class=\"SimplePara\"\u003e25.6\u003c/div\u003e \u003c/td\u003e \u003ctd align=\"left\" colname=\"c8\"\u003e \u003cdiv class=\"SimplePara\"\u003e\u003cspan type=\"BoldItalic\" class=\"BoldItalic\" name=\"Emphasis\"\u003ep\u0026thinsp;\u0026lt;\u0026thinsp;0.001\u003c/span\u003e\u003c/div\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003ctfoot\u003e \u003ctr\u003e\u003ctd colspan=\"8\"\u003e\u003cspan type=\"Italic\" class=\"Italic\" name=\"Emphasis\"\u003e*p\u0026thinsp;\u0026lt;\u0026thinsp;0.05, **p\u0026thinsp;\u0026lt;\u0026thinsp;0.01, ***p\u0026thinsp;\u0026lt;\u0026thinsp;0.001. p-value reached from chi-square test\u003c/span\u003e\u003c/td\u003e\u003c/tr\u003e \u003c/tfoot\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eThere is a significant similarity in the opinions expressed by year-3 and year-5 students about MCQ tests. They did not consider MCQ tests any better in assessing knowledge compared to VSAQ. Over 62% (year-3) and 53% (year-5) students considered BAQ easy to answer, as they contained many distractors easy to exclude as the answer. A good number of students, more of them in year-5 group, had encountered confusing MTF and BAQ items in their examinations.\u003c/p\u003e \u003cp\u003e \u003cspan type=\"Bold\" class=\"Bold\" name=\"Emphasis\"\u003eLecturers\u0026rsquo; feedback on MTF, BAQ and VSAQ\u003c/span\u003e \u003c/p\u003e \u003col\u003e\n\u003cli\u003eMore than 90% were involved in constructing MCQ and BAQ, but less than 10% of them mentioned it was a pleasant experience.\u003c/li\u003e\n\u003cli\u003eMore than 90% felt writing BAQ was more difficult than writing MTF. Over 67% considered writing 3-4 plausible distractors a difficult task.\u003c/li\u003e\n\u003cli\u003eIn MTF, 87% considered writing of false options more difficult than writing true options. Twenty per cent of them said they converted true options to false, and some said they just wrote false options without care, as they were false anyway.\u003c/li\u003e\n\u003cli\u003eAbout using item analysis to improve the questions, 60% mentioned they did not do it.\u003c/li\u003e\n\u003cli\u003eCent per cent respondents said they got ideas about the students\u0026rsquo; strengths and weaknesses while marking directly asked-directly answered questions like MEQ and SAQ, and 40% of them said they do not get any such information from MTF and BAQ results. However, 97% believed such information was important to guide teaching.\u003c/li\u003e\n\u003cli\u003eAbout their openness to VSAQ, 60% answered \u0026lsquo;surely\u0026rsquo; and 40% answered \u0026lsquo;maybe\u0026rsquo;.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Discussion","content":" \u003cp\u003eStudents scored poorly in MTF with high failure rates and very few high scorers. On the contrary, BAQ was skewed towards high scores with very few poor performers. The VSAQ scores showed a fair and balanced distribution as would be expected in a reliable test. There was a strong positive correlation between VSAQ and MTF both in year-3 and year-5 tests (p\u0026thinsp;\u0026lt;\u0026thinsp;0.001), while the correlation between VSAQ and BAQ in both tests were weakly positive (p\u0026thinsp;\u0026lt;\u0026thinsp;0.01). This discrepancy was somewhat reflected in the item analysis of these tests. While most MTF items fell in the moderate difficulty category, BAQ items were more easy and VSAQ more balanced in distribution. Items with 0.2 or higher DISi were more in VSAQ compared to MTF and BAQ, while items with lower DISi were more in both MTF and BAQ tests. Our overall results showed fairer and balanced student performance and superior psychometric properties of VSAQ compared to MTF and BAQ.\u003c/p\u003e \u003cp\u003eStudents\u0026rsquo; opinions about MCQ were generally unfavourable. Mention of many confusing items in the questions was notable. Negative opinions about MTF and BAQ were more evident in year-5 students\u0026rsquo; feedback, which reflected their seniority and the number of tests they would have taken compared to year-3 students. Generally, students\u0026rsquo; opinions were highly in favour of VSQ. Hift [\u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e] proposed elimination of open-ended questions like MEQ in summative assessments, as context-rich well written best answer questions had higher reliability and validity than MEQ. But this author also supported open-ended questions for formative assessments or assessments for learning. Open-ended questions were favoured also by college students [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eValidity and reliability of tests depend on how widely the subject is covered in the assessment [\u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e5\u003c/span\u003e, \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e], but unduly prolonged tests would be counterproductive. Our VSAQ test with 53 questions and 14-item feedback was completed in 30\u0026ndash;45 minutes. MTF of 20 questions took 50 minutes and BAQ of 15 questions took 45 minutes to complete. Our study showed that wider coverage of topics was easier to achieve in VSAQ tests compared to MCQ, as claimed in literature[\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. Test-wiseness and guessing, which are unavoidable in MCQ tests are of no concern in VSAQ. It has also been observed that students found false options harder than true ones to answer, and omission rates due to uncertainty are high in MTF [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]. The adverse effect of using false options was also highlighted by Wood [\u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. BAQ claims higher-order assessment than MTF but can be too easy because of the many non-functioning distractors most of them contain [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eAssessments are of immense value in letting students know what is important to learn [\u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e6\u003c/span\u003e] and for providing feedback to students and directing the teaching [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e, \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e]. VSAQ was seen to achieve these ends better than MCQ. Reading the VSAQ answers while marking provides the examiner's insight into students\u0026rsquo; strengths and deficiencies, which would help in providing feedback and modifying teaching [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e]. MCQ tests do not reveal with certainty what the students know, how much they know, and whether they are capable of using the knowledge in real-life situations, as they do not require the students to construct the answers [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Directly answered questions like VSAQ is an alternative, which will reveal more of students\u0026rsquo; competence. Factual knowledge, important for a doctor [\u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e15\u003c/span\u003e] was effectively tested by our VSAQ tests. Higher taxonomy testing is also said to be better achieved in VSAQ [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. VSAQs are easier to construct [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e]. The struggle of writing MCQ was reflected in our academics\u0026rsquo; feedback. Most of them considered writing good MTF and 3 to 4 plausible distractors in BAQ a difficult task, just as described in the literature [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e]. Our students\u0026rsquo; feedback highlighted the issue of confusing items in MCQ, while no such opinions were expressed about VSAQ. The requirement to construct the answers would drive better learning, as preparation strategies would change [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e10\u003c/span\u003e]. The motivation for learning will be better when students are required to express their knowledge in the examination [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e]. Direct questions let students express their ideas [\u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e], which is a desired purpose of assessments. MCQ can be considered an assessment of convenience far from real-life medical practice, while direct questions are considered more natural [\u003cspan citationid=\"CR2\" class=\"CitationRef\"\u003e2\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eIn the past, when essay questions were used, the onus of constructing the answers was on students. With the advent of MCQ, the roles reversed. Now, the teachers need to cover the syllabus in questions, while the students do not need to write a single word, but to choose the correct answers either knowingly, by guessing or using test-wiseness. In effect, a switch from MCQ to VSAQ will transfer the burden of writing all the answers (true, false and distractors) from the teachers back to the students. Constructing questions will still be an expert job, albeit less arduous than MCQ, as it won\u0026rsquo;t require to create false options and plausible distractors. The only disadvantage envisaged in adopting VSAQ is the time spent on manual scoring. The time spent on constructing MCQ can now be utilised for marking scripts with more rewarding benefits. VSAQ can be scored by anyone with the help of model answers [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e], and the possibility of computer technology [\u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e3\u003c/span\u003e] making the task easier in future is real. Our academics and students responded positively to a shift from MCQ to VSAQ. The demerits of MCQ are many, and there is strong backing for VSAQ, but the transition, as usual, is slow to come.\u003c/p\u003e \u003cp\u003eLimitation of the study\u003c/p\u003e \u003cp\u003eThis study was limited to one institution and the student scores and comparisons were limited to one batch of students each of year-3 and year-5.\u003c/p\u003e "},{"header":"Conclusion","content":" \u003cp\u003eVSAQ employ directly asked questions, which require students to answer briefly and directly with no scope of guessing or reaching the answer by elimination. A larger quantity of knowledge recall, understanding and application can be tested in a shorter period compared to MCQ. The information obtained by the teachers while scoring will help to modify the teaching and to give feedback to the students. Previously used MTF and BAQ items can be converted to VSAQ effectively. Our study showed fairer student performance and better psychometric indexes in VSAQ compared to MTF and BAQ. The students by and large preferred VSAQ to MTF and BAQ. Our academics also expressed the deficiencies of MCQ and showed openness to VSAQ. Last but not the least VSAQ using computers will cut the costs and the carbon footprint of the faculty drastically. In conclusion, VSAQ will be a viable alternative to MCQ with many plus points.\u003c/p\u003e "},{"header":"Abbreviations ","content":"\u003cp\u003eBAQ\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Best answer questions\u003c/p\u003e\n\u003cp\u003eDIFi\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Difficulty index\u003c/p\u003e\n\u003cp\u003eDISi\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Discrimination index\u003c/p\u003e\n\u003cp\u003eIRB\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Institutional Review Board\u003c/p\u003e\n\u003cp\u003eMCQ\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Multiple Choice Questions\u003c/p\u003e\n\u003cp\u003eMTF\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Multiple true-false questions\u003c/p\u003e\n\u003cp\u003eVASQ\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Very Short Answer Questions\u003c/p\u003e\n\u003cp\u003eUNIMAS\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp; Universiti Malaysia Sarawak\u003c/p\u003e"},{"header":"Declarations ","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgements\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur thanks to the dean and deputy deans of the Faculty of Medicine and Health Sciences, UNIMAS for the approval of the project and permission to use the examination questions, results and item analysis reports for this project.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eEthics approval\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe Technical Review Committee approved the study proposal of the Faculty of Medicine and Health Sciences, Universiti Malaysia of Sarawak (UNIMAS). Ethical clearance was also obtained from the Institutional Review Board (IRB) of the Faculty. Strict anonymity and confidentiality of the data were maintained. No student particulars were extracted and disclosed publicly. Students\u0026rsquo; participation in this study was voluntary. The participants signed a formal consent form as per the faculty\u0026rsquo;s ethics committee requirements. The scores of these tests were not used for any formal assessment.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent to publish\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBoth authors read the manuscript and consented for publication\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThe data and materials\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe data will not be available publicly. The principal author keeps all the materials and data used in this study. However, the printed questionnaire for this could be available with permission from the first author.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConflicts of interest\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThere is no conflict of interest involved in this project.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eSelf-funded\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthors\u0026rsquo; contributions\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTP conceptualised the study, captured data and wrote the manuscript, MMR did the statistics, proofreading and editing of the manuscript.\u0026nbsp;\u003c/p\u003e\n"},{"header":"References ","content":"\u003col\u003e\n\u003cli\u003eCirn JT. True/False versus Short Answer Questions. Coll Teach. 1986;34:34\u0026ndash;7. https://www.jstor.org/stable/27558154. Accessed 10 Jan 2020.\u003c/li\u003e\n\u003cli\u003ePickering SG. Against Multiple Choice Questions. Med Teach. 1979;1:84\u0026ndash;6. doi:10.3109/01421597909019397.\u003c/li\u003e\n\u003cli\u003eSam AH, Hameed S, Harris J, Meeran K. Validity of very short answer versus single best answer questions for undergraduate assessment. BMC Med Educ. 2016;16:266.\u003c/li\u003e\n\u003cli\u003evan der Vleuten CPM, Schuwirth LWT. Assessing professional competence: from methods to programmes. Med Educ. 2005;39:309\u0026ndash;17.\u003c/li\u003e\n\u003cli\u003eBurton RF. Multiple‐choice and true/false tests: myths and misapprehensions. Assess Eval High Educ. 2005;30:65\u0026ndash;72. doi:10.1080/0260293042003243904.\u003c/li\u003e\n\u003cli\u003eCase SM, Swanson DB. Constructing Written Test Questions For the Basic and Clinical Sciences. Natl Board Med Exam. 2001;:181. http://www.medbev.umontreal.ca/docimo/DocSource/NBME_MCQ.pdf.\u003c/li\u003e\n\u003cli\u003eHauer KE, Boscardin C, Brenner JM, van Schaik SM, Papp KK. Twelve tips for assessing medical knowledge with open-ended questions: Designing constructed response examinations in medical education. Med Teach. 2019;:1\u0026ndash;6.\u003c/li\u003e\n\u003cli\u003eMartinez ME, Lipson JI. Assessment for Learning. Educ Leadersh. 1989;46:73\u0026ndash;5.\u003c/li\u003e\n\u003cli\u003eVan Der Vleuten CPM, Schuwirth LWT, Driessen EW, Govaerts MJB, Heeneman S. Twelve Tips for programmatic assessment. Med Teach. 2015;37:641\u0026ndash;6.\u003c/li\u003e\n\u003cli\u003eWood T. Assessment not only drives learning, it may also help learning. Med Educ. 2009;43:5\u0026ndash;6.\u003c/li\u003e\n\u003cli\u003eWass V, Van der Vleuten C, Shatzer J, Jones R. Assessment of clinical competence. Lancet Lond Engl. 2001;357:945\u0026ndash;9.\u003c/li\u003e\n\u003cli\u003eChan CKY. Assessment: Short answer question, assessment resources,. 2009. https://ar.cetl.hku.hk/am_saq.htm. Accessed 12 Jan 2020.\u003c/li\u003e\n\u003cli\u003eMcCoubrie P. Improving the fairness of multiple-choice questions: a literature review. Med Teach. 2004;26:709\u0026ndash;12.\u003c/li\u003e\n\u003cli\u003eSim S-M, Rasiah RI. Relationship between item difficulty and discrimination indices in true/false-type multiple choice questions of a para-clinical multidisciplinary paper. Ann Acad Med Singapore. 2006;35:67\u0026ndash;71.\u003c/li\u003e\n\u003cli\u003eAnderson J. For Multiple Choice Questions. Med Teach. 1979;1:37\u0026ndash;42. doi:10.3109/01421597909010580.\u003c/li\u003e\n\u003cli\u003eDutke S, Barenberg J. Easy and Informative: Using Confidence-Weighted True\u0026ndash;False Items for Knowledge Tests in Psychology Courses. Psychol Learn Teach. 2015;14:250\u0026ndash;9. doi:10.1177/1475725715605627.\u003c/li\u003e\n\u003cli\u003ePuthiaparampil T. Assessment Analysis: How It Is Done. MedEdPublish. 2017;6. doi:10.15694/mep.2017.000142.\u003c/li\u003e\n\u003cli\u003eHift RJ. Should essays and other \u0026ldquo;open-ended\u0026rdquo;-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ. 2014;14:249. doi:10.1186/s12909-014-0249-2.\u003c/li\u003e\n\u003cli\u003eStone HL. Return to \u0026ldquo;basics\u0026rdquo; in medical education: a commentary. Med Teach. 1982;4:102\u0026ndash;3.\u003c/li\u003e\n\u003cli\u003ePuthiaparampil T, Gudum HR, Rahman MM, Saimon R, Lim IF. True-false analysis reveals inherent flaws in multiple true-false tests. Int J Community Med Public Health. 2019;6:4204\u0026ndash;8. doi:10.18203/2394-6040.ijcmph20194476.\u003c/li\u003e\n\u003cli\u003eTarrant M, Ware J, Mohammed AM. An assessment of functioning and non-functioning distractors in multiple-choice questions: a descriptive analysis. BMC Med Educ. 2009;9:40. doi:10.1186/1472-6920-9-40.\u003c/li\u003e\n\u003cli\u003eCase SM, Swanson DB. Constructing Written Test Questions For the Basic and Clinical Sciences. 1998;:181.\u003c/li\u003e\n\u003c/ol\u003e\n"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Very Short Answer Questions, Multiple Choice Questions, Best answer questions","lastPublishedDoi":"10.21203/rs.3.rs-16617/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-16617/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eBackground Multiple choice questions, used in medical school assessments for decades, have many drawbacks, such as: hard to construct, allow guessing, encourage test-wiseness, promote rote learning, provide no opportunity for examinees to express ideas, and do not provide information about strengths and weakness of candidates. Directly asked and answered questions like Very Short Answer Questions (VSAQ) is considered a better alternative with several advantages.\u003c/p\u003e\u003cp\u003e Objectives This study aims to substantiate the superiority of VSAQ by actual tests and obtaining feedback from the stakeholders. \u003c/p\u003e\u003cp\u003eMethods Conduct multiple true-false, one best answer and VSAQ tests in two batches of medical students, compare their scores and psychometric indexes of the tests and seek opinions from students and academics regarding these assessment methods. \u003c/p\u003e\u003cp\u003eResults Multiple true-false and best answer test scores showed skewed results and low psychometric performance compared to better psychometrics and more balanced student performance in VSAQ tests. The stakeholders’ opinions were significantly in favour of VSAQ. \u003c/p\u003e\u003cp\u003eConclusion and recommendation This study concludes that VSAQ is a viable alternative to multiple choice question tests, and it is widely accepted by medical students and academics in the medical faculty.\u003c/p\u003e","manuscriptTitle":"Very Short Answer Questions: A Viable Alternative to Multiple Choice Questions","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2020-03-10 21:16:29","doi":"10.21203/rs.3.rs-16617/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"editorInvitedReview","content":"","date":"2020-04-07T12:00:00+00:00","index":2,"fulltext":"Recommendation: Reject\nForm responses:\n---\n\nComments to Author:\n---\nI will not comment directly on the abstract, as it will become evident from my comments below that the abstract needs rewriting.\n\nYour objective sets out to compare the formats - but your abstract asserts a superiority for VSAQ. The research should test the null hypothesis but appears predicated on a prejudged outcome.\n\nMETHOD\nThe method is very short. A clear method will allow me to conduct the same study myself. In its present form, I cannot do so. There are a number of queries posed by your method:\n1 You don't provide any detail on how students were enrolled, the inclusion criteria, their demography or their known academic performance. Were there any exclusion criteria?\n2 I am unclear on how the questions were selected, since each format seems to use a different set of questions. What assurance did you have about the quality, discriminatory value and prior performance of the questions.\n3 How did you know the questions tested the same knowledge?\n4 You generated VSAQ by removing the answer options from BAQ format (Ill return to this later), but were any of them used in both formats?\n5 You don't say whether answers not on the grid were permitted or not (and, if not, why not). Our experience suggest that VSAQ can lead to unexpected correct answers.\n6 You don't make clear whether these are the same standard of questions used for both yr groups (as in progress testing) or aligned to the standard for that year of study.\n\nRESULTS\nI find your results comparing the performance of the formats curious and unexpected. Sam et al in their earlier paper (cited) and in a subsequent paper in medical education (see below for ref) showed performance in BAQ (SBA) format to be higher than VSAQ (VSA). Later in this section you state 'students considered BAQ easy to answer as they contained many distractors easy to exclude as the answer'. It is acknowledged that creating realistic distractors is a challenge for question writers, but this suggests that the BAQ quality of questions was less good and may explain why students did similarly well in both formats.\nYou have 'chunked' the pass marks into groups but do not explain why you have chosen these groups as opposed to any other stratification. Nor how many students feature in each group. You state the performance of the VSAQ 'shows a desirable balanced distribution' but this should be based on criterion referencing and so will depend on the academic capability of the group of students being tested.\n\nDISCUSSION\nYour discussion on the findings of the test performance seems to be predicated on the assumption that VSAQ are 'more balanced' and consists of a number of comparative statements without a consideration of what the findings might mean or how they are explained. For example, why should VSA perform as well as BAQ given the former depends on recall/application of knowledge whereas the latter can be simply a matter of potential (pattern) recognition (and easier because 'they contained many distractors easy to exclude in the answer'). I am equally puzzled to to why VSAQ should be done more quickly than BAQ since 1/ a good question should require more synthesis by the student to derive an answer and 2/ particularly if there are poor distractors, it should be quicker to arrive at the answer. It seems to me that there is a need to explore/consider the quality of the question writing in the first place as a factor in explaining your findings.\n\nI haven't commented on the survey findings but note this is consistent with the findings of previous research in repeat of the VSAQ format.\n\nCONCLUSIONS\nYour conclusions match the your paper findings, but don't seem to add anything to what is already published (see also below)\n\nand finally\nTwo relevant and more recent papers from Sam et al that are pertinent to this topic will help you think through your approach to this paper. doi:10.1111/medu.13504 and doi:10.1136/bmjopen-2019-032550\n\n* Are the methods appropriate and well described?: **No**\n* Does the work include the necessary controls?: **Unable to assess**\n* Are the conclusions drawn adequately supported by the data shown?: **No**\n* Are you able to assess any statistics in the manuscript or would you recommend an additional statistical review?: **I recommend additional statistical review**\n* Quality of written English: **Acceptable**\n* Declaration of competing interests: **I have published on Very Short Answers as a co-author of several recent publications**\n* Reviewer Publication Consent. I agree for my report to be made available under an Open Access Creative Commons CC-BY License (http://creativecommons.org/licenses/by/4.0) if this manuscript is accepted for publication. Any comments that I do not wish to be included in the published report have been included as confidential comments to the editor, which will not be published.: **I agree to the terms of the CC-BY 4.0 license; please do not publish my name with my report. (default)**\n"},{"type":"decision","content":"Major revision","date":"2020-04-07T12:00:00+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-03-24T12:00:00+00:00","index":2,"fulltext":""},{"type":"editorInvitedReview","content":"","date":"2020-03-24T12:00:00+00:00","index":1,"fulltext":"Recommendation: Accept after discretionary revisions\nForm responses:\n---\n\nComments to Author:\n---\nPage 2, Line 3-many drawbacks such as (unnecessary comma and colon may be avoided)\nPage 2, Line 20-psychometric indices\nPage 2, Line 41- replace 'in' with 'by the'\nPage 3, Line 25-replace feedbacks with feedback\nMethodology-The study design, duration, setting has not been mentioned\nThe validation of questionnaire has not been mentioned, if it is a validated one reference has not been cited\nReference for source of MCQs have not been mentioned\nTime taken for actual tests must be mentioned here not in discussion\nScoring pattern of MCQ, VSAQ and BAQ not mentioned\nUse of Independent T test as shown in table 1 and chi square as in Figure 1 not mentioned\nHow item analysis was performed mentioned in the results...it should be placed in methodology\nDiscrimination index calculation is not mentioned\nResults\nDemography not mentioned\nIn figure 1-x axis is % marks not index, y axis-% of students (not shown), Pearson is wrongly spelt\nPage 4, Line 46- however those scoring \u003e75% declined in contrast to BAQ (this observation may be added)\nPage 5, Line 40- citation reference not shown\nPage 6, Line 35-UNIMAS expansion to be provided\nPage 6, Line 47-Year 5- percentages add upto 94.9(84.6+10.3)-please recheck\nPage 6, Line 52-Year 5- 59+33.3+17.9=110.2...please recheck\nPage 6, Line 58- Year 3-Percentages 67.6+24.3+13.5 add to 105.4 ..please recheck\nPage 6, Line 58-Percentage 87.2+7.7+12.8=107.7, please recheck\nPage 7, Line 5-Percentage 84.6+12.8=97.4, please recheck\nPage 7,Table 3- Many percentages add to 99.9 and 100.1 instead of 100, please correct\nPage 8, Line 27- replace MCQ with MTF as it is more suitable here\nPae 8, Line 39-MEQ-elaborate abbreviation\nPage 9, Line6-was to be replaced by were\nPage 9- Lines 17-20 on time taken to answer questions may be placed in methodology than in discussion\nPage 9, Line 25-indices instead of indexes\nPage 10-Majority of the references are a bit old(\u003e15 years old)\n* Are the methods appropriate and well described?: **No**\n* Does the work include the necessary controls?: **Yes**\n* Are the conclusions drawn adequately supported by the data shown?: **Yes**\n* Are you able to assess any statistics in the manuscript or would you recommend an additional statistical review?: **I am able to assess the statistics**\n* Quality of written English: **Needs some language corrections before being published**\n* Declaration of competing interests: **I declare that I have no competing interests**\n* I agree to the open peer review policy of the journal. I understand that my name will be included on my report to the authors and, if the manuscript is accepted for publication, my named report including any attachments I upload will be posted on the website along with the authors' responses. I agree for my report to be made available under an Open Access Creative Commons CC-BY license (http://creativecommons.org/licenses/by/4.0/). I understand that any comments which I do not wish to be included in my named report can be included as confidential comments to the editors, which will not be published.: ** I agree to the open peer review policy of the journal**\n"},{"type":"editorAssigned","content":"","date":"2020-03-12T12:00:00+00:00","index":"","fulltext":""},{"type":"reviewersInvited","content":"","date":"2020-03-12T12:00:00+00:00","index":"","fulltext":""},{"type":"reviewerAgreed","content":"","date":"2020-03-12T12:00:00+00:00","index":1,"fulltext":""},{"type":"checksComplete","content":"","date":"2020-03-05T12:00:00+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2020-03-05T12:00:00+00:00","index":"","fulltext":""},{"type":"submitted","content":"","date":"2020-02-24T12:00:00+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"2275fe94-e32d-4583-bd60-ea932c2ca502","owner":[],"postedDate":"March 10th, 2020","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[{"id":68204,"name":"Educational Philosophy and Theory"},{"id":68205,"name":"Internal Medicine"}],"tags":[],"updatedAt":"2021-07-27T21:00:02+00:00","versionOfRecord":{"articleIdentity":"rs-16617","link":"https://doi.org/10.1186/s12909-020-02057-w","journal":{"identity":"bmc-medical-education","isVorOnly":false,"title":"BMC Medical Education"},"publishedOn":"2020-05-06 21:00:02","publishedOnDateReadable":"May 6th, 2020"},"versionCreatedAt":"2020-03-10 21:16:29","video":"","vorDoi":"10.1186/s12909-020-02057-w","vorDoiUrl":"https://doi.org/10.1186/s12909-020-02057-w","workflowStages":[]},"version":"v1","identity":"rs-16617","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-16617","identity":"rs-16617","version":["v1"]},"buildId":"B-jG_2CBjPDmsCi4Wdhf-","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}