Student Evaluations of Teaching Do Not Reflect Student Learning: An Observational Study

Student Evaluations of Teaching Do Not Reflect Student Learning: An Observational Study | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Student Evaluations of Teaching Do Not Reflect Student Learning: An Observational Study Robert O Gilbert, Dave R Gilbert This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4224772/v1 This work is licensed under a CC BY 4.0 License Status: Published Journal Publication published 26 Feb, 2025 Read the published version in BMC Medical Education → Version 1 posted 10 You are reading this latest preprint version Abstract Background: Student Evaluations of Teaching (SET) are routinely utilized in the assessment of university faculty for purposes of hiring, promotion, tenure, and merit-based salary increases. Despite their ubiquity, much evidence exists that they are biased, amongst other factors by expected and received student grades. To our knowledge, this issue has not been examined in veterinary education. Furthermore, it remains unclear whether the combination of higher grades and more favorable student evaluations might reflect enhanced learning. Our study evaluates the relationship between (A) student evaluations of courses in a veterinary curriculum, (B) grades earned in those courses, and (C) independent measures of learning in those subjects. Methods: The Veterinary Educational Assessment (VEA) is an independent, external exam in basic sciences subjects administered by the International Council for Veterinary Assessment and is taken by Ross University School of Veterinary Medicine (RUSVM) students in their fifth semester of study. It offers an external means of measuring student learning in specific subjects and relating them to course evaluations. RUSVM has three terms each year with three separate intakes of students. Course evaluations and student grades were recorded for courses from fall 2018 to summer 2022, spanning 12 cohorts of students, and 152 individual courses. Courses were aligned to the relevant section of the VEA taken by each cohort. Spearman correlation coefficients were calculated. Results: Mean course evaluations were significantly positively correlated to median grade in the course (rho = 0.33, P < 0.0001) and the proportion of students earning A-grades (rho = 0.35, P < 0.0001). The relationship between course evaluation and relevant VEA score was negative (rho = -0.18, P = 0.027), indicating that students judged courses favorably when higher grades were expected without necessarily learning more from those courses. Conclusions: We confirmed the well-known relationship between SET and student grades but, for the first time in veterinary medicine, describe a small but negative relationship between SET and an independent measure of learning. SET should be interpreted with caution; their use for evaluation of teachers may have unintended consequences including reduced expectations for student achievement. Student evaluation of teaching Veterinary education Veterinary Educational Assessment Figures Figure 1 Figure 2 Background Student evaluations of teaching (SET) are now a standard tool in post-secondary education to evaluate the teaching effectiveness of faculty. They inform decisions regarding wage increases, promotion, and tenure. A survey of four-year, liberal arts colleges in the United States found that 94% “always used” SET in assessing faculty teaching.(1) SET tend to be conducted at the end of a course and ask students to evaluate their instructor(s) as well as the course overall. Students are asked to indicate their level of agreement with statements like “The instructor was available to students outside of class”, “Overall, I learned a lot from this instructor”, or “The course was well planned”, and responses are typically recorded via a 5-point Likert scale, with options ranging from “Strongly disagree” to “Strongly Agree” (or similar). In addition, students may be asked more general questions, such as “Overall, how would you rate this course/instructor?” Again, responses are collected via a 5-point Likert scale. Implicit in the perceived value of SET is the assumption that students’ reported perceptions of teaching quality are an accurate measure of actual teaching quality: if SET are to be used as a basis for promotion or merit, they should be a reliable measure of teaching effectiveness. There is a significant body of evidence that this assumption is not justified. Research on SET has suggested the presence of biases according to, amongst other factors, gender, physical attraction, race, ethnicity, age, culture, likability, and prior interest in the subject matter. (2-8) In addition, and most relevant to our present purposes, studies repeatedly demonstrate a positive correlation between high grades (or expected grades) in courses and SET. This correlation has been interpreted by some as validating SET: higher marks are indicative of increased learning and, therefore, good teaching.(9, 10) On this interpretation, good teaching leads to student learning, which, in turn, leads to increased course grades and higher SET. Following Johnson, one might call this explanation the teacher-effectiveness theory .(11) On the other hand, the correlation has been attributed to a biasing effect “in the sense that the effect of grading on teaching evaluations represents a factor not related to either effective teaching or student learning.”(11) Though different mechanisms have been posited to explain the precise nature of the bias, the common feature among grade-bias theories is that the expectation of higher grades—distinct from high-quality teaching—drives more favorable SET.(12, 13) Several recent multi-section validity studies and meta-analyses seem to indicate that the teacher-effectiveness theory is not a good explanation of the observed positive correlation, lending support to the grade-bias theories.(14-16) To our knowledge, there are no reports on these issues in the context of veterinary education. With regard to medical education, the known biases of SET and the difficulties involved in relating student evaluations to actual learning have previously been noted, as has the basic dichotomy of (medical) students rating courses on the basis of their ability to pass exams whereas their teachers seek more broadly to prepare them to become doctors in the context of a comprehensive curriculum.(17) At least one attempt to relate the extant literature on SET to veterinary education seems to endorse the effectiveness theory.(18) However, the literature has also been interpreted to support the grade-bias theory.(12) It seems that no specific data exist to interrogate this question specifically in the context of veterinary education. The aim of this paper is to evaluate the relationship between SET for courses in a veterinary curriculum, grades earned in those courses, and independent measures of learning in those subjects. In addition to being focused on veterinary education in particular, the use (and availability) of an independent, external measure of learning distinguishes the current study from others in the literature. Methods This study was reviewed by the IRB of Ross University School of Veterinary Medicine (RUSVM) and deemed to be exempt from review and from individual student consent in accordance with U.S. Federal Regulations 45 CFR 46.102. reference 24-03. RUSVM has three academic terms each year. Intake of new students occurs tri-annually as well, coordinated with the beginning of the academic terms. During the period of this study, entering classes numbered 180 to 200 students. Attrition over the first four semesters was approximately 15%. All students took all courses in the same sequence. SET during this period were conducted at the end of each course. Students had electronic access to the survey instrument beginning about 3 weeks before the end of term and continuing until two weeks after the end of the term. Therefore, some students would have completed the survey before knowing their final grades and others would have known the final outcome of each course. The Veterinary Educational Assessment (VEA) is an independent, standardized, external exam in basic sciences subjects administered by the International Council for Veterinary Assessment in cooperation with the National Board of Medical Examiners and is taken by RUSVM students in their fifth semester of study. Students take a 240-item, web-based, multiple-choice exam with sections on anatomy, physiology, pharmacology, microbiology, and pathology. Courses were aligned with components of the VEA as depicted in Table 1 below: Table 1. Alignment of Courses taught and VEA sections VEA Section Course Subject titles Anatomy Gross Anatomy I, Gross Anatomy II, Microscopic Anatomy & Embryology Physiology Physiology I, Physiology II Microbiology Principles of Infectious Diseases, Immunology, Parasitology, Bacteriology, Virology, Veterinary Public Health Pathology Pathology I, Pathology II, Clinical Pathology Pharmacology Pharmacology We recorded SET, course grades, and the percentage of students with grades in the A-range for 152 individual courses between the fall 2018 and summer 2022 terms: the first four terms of study for 12 cohorts of students. Each course was aligned with the relevant section of the VEA taken by each cohort. The categories/questions according to which students were asked to evaluate courses were: The course was well planned, organized and followed a coherent pattern. Learning activities used in this course [lectures, small groups, demonstrations, labs, case discussions, quizzes, etc.] were effective. The course was intellectually challenging and stimulating. Exam questions for this course challenged me to critically think about concepts I was presented. The examinations were administered according to the Student Handbook. The course conformed to the schedule published in the syllabus. The amount of time I invested in this course was appropriate for the allotted credits. Rate the course overall. We explored the relationships between component questions as well as those between the questions and VEA results in order to probe the question of teacher-effectiveness versus grade bias. Course evaluations were not normally distributed. Therefore, Spearman correlation coefficients were calculated to explore the relationships between the median grade for the course, the mean SET, the percentage of A-grades, and VEA score. We also explored the relationship between the individual questions in the SET. (Notwithstanding the well-known problems involved with treating data derived from Likert scales as continuous, we used the SET means because that is what is commonly used at many universities and colleges.) Results Mean course evaluations were significantly positively related to median grades in the courses (rho = 0.33, P = 0.0001). In addition, they were significantly positively correlated with the proportion of students earning final grades in the A-range (rho = 0.35, P < 0.0001). On the other hand, the relationship between course evaluations and the corresponding sectional VEA scores was negative (rho = -0.18, P = 0.027), as was the correlation between the VEA scores and the percentage of grades in the A-range (rho = -0.23, P < 0.0043). These data are summarized in Table 2. Table 2. Relationship of Response to "rate the course overall" to course median grade and VEA scores. (Cells contain Spearman's correlation coefficient (rho), P-value, and number of courses.) Overall Evaluation Median Grade Percent of A-grades VEA score Overall Evaluation 1 Median Score 0.33 0.0001 152 1 Percent of A-grades 0.35 0.0001 152 0.84 0.0001 152 1 VEA score -0.18 0.027 152 -0.14 0.075 152 -0.27 0.0007 152 1 The relationships between median course grades and means SET score (positive) are shown in Figure 1. Figure 2 shows the scatter plot and trend line for the negative relationship between mean course evaluation and VEA score. Notably, responses to all component questions of the student evaluation were significantly correlated to each other. Spearman correlation coefficients ranged from 0.67 to 0.92, and for all P 0.05) or negatively correlated. All were significantly and positively correlated with course median grade and with the proportion of the class earning grades in the A-range. Even responses to the question “The course was intellectually challenging and stimulating” were positively correlated to course median and mean grades. In other words, courses were rated as more challenging when grades were higher, in contrast to what might be expected (in that more challenging courses might be expected to yield lower grades). Even more strikingly, student assessment of whether or not courses adhered to the published syllabus also varied with the median grade—even though essentially no courses deviated at all from the syllabus and, if they had, it would have constituted a basis for grade appeals and formal complaints. Relatedly, the fact that students were more likely to rate “time invested in the course” as appropriate for the allocated credit hours if they received or anticipated high grades despite the negative correlation of this component with actual learning measured by VEA scores—implying that students favored courses that offered higher grades without great expenditure or effort—adds further support to grade-bias interpretations. Discussion Our study confirms the well-established grade bias to which SET is subject. For the first time in a veterinary educational context, the negative relationship between SET and actual learning is illustrated. A novel aspect of the study is that we were able to take advantage of the VEA, as an external and independent measure of learning to explore the relationship between SET and learning. While the negative relationship between SET and VEA scores is not strong, it is statistically significant and clearly undermines any argument that favorable SET and high course grades both reflect superior teaching and, by implication, learning. This, in turn, argues for extreme caution in the use of SET for informing important, career-defining decisions such as faculty remuneration, promotion or tenure. Importantly, with regard to the design of this study, instructors of the included courses had control over the content, pace, methods of assessment, and course design more generally. Hence, our study is not obviously subject to a criticism that has been made of multi-section validity studies: that wresting control of courses from instructors and placing it in the hands of study designers undermines the quality of the courses, obfuscates the relevance of any observed correspondence between evaluations and marks, and reduces the scope of any conclusions.(11) The consistent correlation between component questions of the SET, and their relationship to grades earned in courses, supports the contention that students seem prone to complete surveys without attention to detail, once having decided their overall opinion of the course. An extreme instance of lack of diligence in completion of course evaluations is a study by Uijtdehaage and O’Neal in which 66% of students completed assessments for a fictitious instructor and 49% did so even when a photograph of the phantom teacher was included.(19) Limitations The lack of a one-to-one relationship between subjects taught and VEA sections is a weakness of this study, but we believe that the availability of an external, objective measure that cannot be manipulated by the instructors or the institution provides a unique opportunity to study student evaluation of teaching and actual learning, offsetting this acknowledged deficit. Secondly, in order to interpret the negative correlation between SET/grades and performance on the external exam as disconfirming evidence of teacher-effectiveness theories, one needs to assume that: ( i ) student learning is an acceptable measure of teaching effectiveness (or a significant component thereof); and ( ii ) measuring performance on the external exam is a reasonable proxy for assessing learning. Proponents of teacher-effectiveness theories might well grant ( i ) but argue, against ( ii ), that course grades are a better indicator of learning than a standardized exam. However, recent studies exploring and modeling the mechanisms relating SET to grade inflation cast some doubt against the validity of such an argument.(20) Our study cannot settle this debate. Conclusion Student evaluations of teaching should be interpreted very cautiously; their use for evaluation of teachers or for salary improvement or promotion may be counter-productive and penalize some of the most effective teachers and provide unintended incentives for teachers to lower expectations of student mastery of material. Abbreviations IRB: Institutional Review Board RUSVM: Ross University School of Veterinary Medicine SET: Student Evaluation of Teaching VEA: Veterinary Educational Assessment Declarations Ethics approval: This study was reviewed by the IRB of Ross University School of Veterinary Medicine and deemed to be exempt from review and from individual student consent in terms of U.S. Federal Regulations 45 CFR 46.102. Consent for publication: Not applicable Availability of data : Data are available from the corresponding author upon reasonable request. Competing interests: The authors declare no competing interests. Funding: None Author Contributions: The authors confirm contribution to the paper as follows: study conception and design: ROG and DRG; data acquisition: ROG; analysis of results: ROG; draft manuscript preparation: ROG and DRG. Both authors reviewed the results and approved the final version of this manuscript. Both authors agree to be accountable for all aspects of the work. Acknowledgements: Not applicable References Miller J, Seldin P. Changing practices in faculty evaluations: can better evaluation make a difference? https://www.aaup.org/article/changing-practices-faculty-evaluation: American Association of University Professors; 2014 [cited 2023 November 22]. Boring A. Gender biases in student evaluations of teaching. Journal of public economics. 2017;145:27-41. Fan Y, Shepherd LJ, Slavich E, Waters D, Stone M, Abel R, et al. Gender and cultural bias in student evaluations: Why representation matters. PLOS ONE. 2019;14(2):e0209749. Hamermesh DS, Parker A. Beauty in the classroom: Instructors pulchritude and putative pedagogical productivity. Economics of education review. 2005;24(4):369-76. Wolbring T, Riordan P. How beauty works. Theoretical mechanisms and two empirical applications on students' evaluation of teaching. Social science research. 2016;57:253-72. Smith BP. Student ratings of teaching effectiveness: an analysis of end-of-course faculty evaluations. College student journal. 2007;41(4):788. Joye S, Wilson JH. Professor Age and Gender Affect Student Perceptions and Grades. The journal of scholarship of teaching and learning. 2015;15(4):126-38. Feistauer D, Richter T. Validity of students’ evaluations of teaching: Biasing effects of likability and prior subject interest. Studies in educational evaluation. 2018;59:168-78. Cohen PA. Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies. Review of educational research. 1981;51(3):281-309. Feldman KA. The Association between Student Ratings of Specific Instructional Dimensions and Student Achievement: Refining and Extending the Synthesis of Data from Multisection Validity Studies. Research in higher education. 1989;30(6):583-645. Johnson VE, Springer LeEIC, Ebook C, SpringerLink. Grade inflation: a crisis in college education. New York: Springer; 2003. Anderson RE, Choi KS, Hair JF. Cognitive Consistency Theory and Student Evaluation of Teacher Effectiveness. The Journal of experimental education. 1975;44(2):64-70. Clayson DE, Frost TF, Sheffet MJ. Grades and the Student Evaluation of Instruction: A Test of the Reciprocity Effect. Academy of Management learning & education. 2006;5(1):52-65. Clayson DE. Student Evaluations of Teaching: Are They Related to What Students Learn?: A Meta-Analysis and Review of the Literature. Journal of marketing education. 2009;31(1):16-30. Boring A, Ottoboni K, Stark P. Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. ScienceOpen research. 2016. Uttl B, White CA, Gonzalez DW. Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in educational evaluation. 2017;54:22-42. Looi JCL, Anderson KJ. Between SET and ASP: balancing the scales of student evaluation of teaching (SET) and teachers’ assessments of student performance (ASP) for medical school education in psychiatry. Australasian psychiatry : bulletin of the Royal Australian and New Zealand College of Psychiatrists. 2018;26(6):659-61. Beran TN, Donnon T, Hecker K. A review of student evaluation of teaching: applications to veterinary medical education. Journal of veterinary medical education. 2012;39(1):71. Uijtdehaage S, O'Neal C. A curious case of the phantom professor: mindless teaching evaluations by medical students. Med Educ. 2015;49(9):928-32. Stroebe W. Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis. Basic and applied social psychology. 2020;42(4):276-94. Additional Declarations No competing interests reported. Cite Share Download PDF Status: Published Journal Publication published 26 Feb, 2025 Read the published version in BMC Medical Education → Version 1 posted Editorial decision: Revision requested 26 May, 2024 Reviews received at journal 22 May, 2024 Reviews received at journal 21 May, 2024 Reviewers agreed at journal 14 May, 2024 Reviewers agreed at journal 14 May, 2024 Reviewers invited by journal 14 May, 2024 Editor assigned by journal 14 May, 2024 Editor invited by journal 08 May, 2024 Submission checks completed at journal 08 May, 2024 First submitted to journal 05 Apr, 2024 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4224772","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":302335536,"identity":"2d627976-458a-4e96-88cd-64c99bc95dc8","order_by":0,"name":"Robert O Gilbert","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABCklEQVRIie2QsWrDMBCGzxjk5UpWGUP7CgqCQKiJX8XCoC6mLWTJFkPAWfQAGfIgGVUMzqIHKHSp6dpCu5QMgVZ1hw6N0owd9E3i+L/jfgF4PP+QrAofdVfZV1T9TClA7lSYJkwLm0bU3xP8W7GRXqH5qcpWBVps0pssfum63b45zyC8e0CQ184uyoAWRo5VcsX5Wd1wBFJcIpRTlzKh5a0WdcMwkSQJKikU4ChBmH1dexBOS1u//mAYtyTe7eVcweD9qDJc9YpmSAmhSNIcAYlVSqfCTGsVUzA0MrRd0qFqCB+vmXQr28XT69tmwqJlG9gfoxfRctHdP88Kp3KAsF91et7j8Xg8v/kE0MFUkddFidcAAAAASUVORK5CYII=","orcid":"","institution":"Ross University School of Veterinary Medicine","correspondingAuthor":true,"prefix":"","firstName":"Robert","middleName":"O","lastName":"Gilbert","suffix":""},{"id":302335537,"identity":"7a92ebfc-8865-422d-9459-e5c61e37dac6","order_by":1,"name":"Dave R Gilbert","email":"","orcid":"","institution":"University of British Columbia","correspondingAuthor":false,"prefix":"","firstName":"Dave","middleName":"R","lastName":"Gilbert","suffix":""}],"badges":[],"createdAt":"2024-04-05 20:44:18","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4224772/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4224772/v1","draftVersion":[],"editorialEvents":[{"content":"https://doi.org/10.1186/s12909-025-06896-3","type":"published","date":"2025-02-26T15:57:45+00:00"}],"editorialNote":"","failedWorkflow":false,"files":[{"id":56678980,"identity":"22a52662-1c3e-41b5-b8ad-785de56c24ff","added_by":"auto","created_at":"2024-05-17 16:40:09","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":21864,"visible":true,"origin":"","legend":"\u003cp\u003eScatter plot and regression line showing relationship between median course grade and meanl course evaluation (P = 0.0001).\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4224772/v1/d28b0dc961d95ed661099b8a.png"},{"id":56678738,"identity":"a532f17e-281a-4650-bdd2-caa25ccac422","added_by":"auto","created_at":"2024-05-17 16:39:31","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":22354,"visible":true,"origin":"","legend":"\u003cp\u003eScatter plot and regression line showing relationship between Overall course evaluation and corresponding VEA scores (P = 0.027).\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-4224772/v1/13d5098dc0b49c4e6b658fb2.png"},{"id":77622707,"identity":"ed169cbe-62cb-4959-9ae9-116e1bb32a6e","added_by":"auto","created_at":"2025-03-03 16:09:36","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":430889,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4224772/v1/dbfb7980-cfbf-430d-92dc-67eaaae63d8a.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Student Evaluations of Teaching Do Not Reflect Student Learning: An Observational Study","fulltext":[{"header":"Background","content":"\u003cp\u003eStudent evaluations of teaching (SET) are now a standard tool in post-secondary education to evaluate the teaching effectiveness of faculty. They inform decisions regarding wage increases, promotion, and tenure. A survey of four-year, liberal arts colleges in the United States found that 94% \u0026ldquo;always used\u0026rdquo; SET in assessing faculty teaching.(1)\u0026nbsp;SET tend to be conducted at the end of a course and ask students to evaluate their instructor(s) as well as the course overall. Students are asked to indicate their level of agreement with statements like \u0026ldquo;The instructor was available to students outside of class\u0026rdquo;, \u0026ldquo;Overall, I learned a lot from this instructor\u0026rdquo;, or \u0026ldquo;The course was well planned\u0026rdquo;, and responses are typically recorded via a 5-point Likert scale, with options ranging from \u0026ldquo;Strongly disagree\u0026rdquo; to \u0026ldquo;Strongly Agree\u0026rdquo; (or similar). In addition, students may be asked more general questions, such as \u0026ldquo;Overall, how would you rate this course/instructor?\u0026rdquo; Again, responses are collected via a 5-point Likert scale. Implicit in the perceived value of SET is the assumption that students\u0026rsquo; reported perceptions of teaching quality are an accurate measure of actual teaching quality: if SET are to be used as a basis for promotion or merit, they should be a reliable measure of teaching effectiveness. There is a significant body of evidence that this assumption is not justified.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Research on SET has suggested the presence of biases according to, amongst other factors, gender, physical attraction, race, ethnicity, age, culture, likability, and prior interest in the subject matter. (2-8) In addition, and most relevant to our present purposes, studies repeatedly demonstrate a positive correlation between high grades (or expected grades) in courses and SET. This correlation has been interpreted by some as validating SET: higher marks are indicative of increased learning and, therefore, good teaching.(9, 10) On this interpretation, good teaching leads to student learning, which, in turn, leads to increased course grades and higher SET. Following Johnson, one might call this explanation the \u003cem\u003eteacher-effectiveness theory\u003c/em\u003e.(11) On the other hand, the correlation has been attributed to a biasing effect \u0026ldquo;in the sense that the effect of grading on teaching evaluations represents a factor not related to either effective teaching or student learning.\u0026rdquo;(11) Though different mechanisms have been posited to explain the precise nature of the bias, the common feature among \u003cem\u003egrade-bias theories\u003c/em\u003e is that the expectation of higher grades\u0026mdash;distinct from high-quality teaching\u0026mdash;drives more favorable SET.(12, 13) Several recent multi-section validity studies and meta-analyses seem to indicate that the teacher-effectiveness theory is not a good explanation of the observed positive correlation, lending support to the grade-bias theories.(14-16)\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;To our knowledge, there are no reports on these issues in the context of veterinary education. With regard to medical education, the known biases of SET and the difficulties involved in relating student evaluations to actual learning have previously been noted, as has the basic dichotomy of (medical) students rating courses on the basis of their ability to pass exams whereas their teachers seek more broadly to prepare them to become doctors in the context of a comprehensive curriculum.(17) At least one attempt to relate the extant literature on SET to veterinary education seems to endorse the effectiveness theory.(18) However, the literature has also been interpreted to support the grade-bias theory.(12) It seems that no specific data exist to interrogate this question specifically in the context of veterinary education. The aim of this paper is to evaluate the relationship between SET for courses in a veterinary curriculum, grades earned in those courses, and independent measures of learning in those subjects. In addition to being focused on veterinary education in particular, the use (and availability) of an independent, external measure of learning distinguishes the current study from others in the literature.\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003eThis study was reviewed by the IRB of Ross University School of Veterinary Medicine (RUSVM) and deemed to be exempt from review\u0026nbsp;and from individual student consent in accordance with U.S. Federal Regulations 45 CFR 46.102. reference 24-03.\u003c/p\u003e\n\u003cp\u003eRUSVM has three academic terms each year. Intake of new students occurs tri-annually as well, coordinated with the beginning of the academic terms. During the period of this study, entering classes numbered 180 to 200 students. \u0026nbsp;Attrition over the first four semesters was approximately 15%. All students took all courses in the same sequence. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSET during this period were conducted at the end of each course. \u0026nbsp;Students had electronic access to the survey instrument beginning about 3 weeks before the end of term and continuing until two weeks after the end of the term. Therefore, some students would have completed the survey before knowing their final grades and others would have known the final outcome of each course. \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe Veterinary Educational Assessment (VEA) is an independent, standardized, external exam in basic sciences subjects administered by the International Council for Veterinary Assessment in cooperation with the National Board of Medical Examiners and is taken by RUSVM students in their fifth semester of study. Students take a 240-item, web-based, multiple-choice exam with sections on anatomy, physiology, pharmacology, microbiology, and pathology.\u003c/p\u003e\n\u003cp\u003eCourses were aligned with components of the VEA as depicted in Table 1 below:\u003c/p\u003e\n\u003cp\u003eTable\u0026nbsp;1. Alignment of Courses taught and VEA sections\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eVEA Section\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003e\u003cstrong\u003eCourse Subject titles\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003eAnatomy\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003eGross Anatomy I, Gross Anatomy II, Microscopic Anatomy \u0026amp; Embryology\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003ePhysiology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003ePhysiology I, Physiology II\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003eMicrobiology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003ePrinciples of Infectious Diseases, Immunology, Parasitology, Bacteriology, Virology, Veterinary Public Health\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003ePathology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003ePathology I, Pathology II, Clinical Pathology\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"15.890850722311397%\" valign=\"top\"\u003e\n \u003cp\u003ePharmacology\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"84.1091492776886%\" valign=\"top\"\u003e\n \u003cp\u003ePharmacology\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eWe recorded SET, course grades, and the percentage of students with grades in the A-range for 152 individual courses between the fall 2018 and summer 2022 terms: the first four terms of study for 12 cohorts of students. Each course was aligned with the relevant section of the VEA taken by each cohort.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe categories/questions according to which students were asked to evaluate courses were:\u003c/p\u003e\n\u003col\u003e\n \u003cli\u003eThe course was well planned, organized and followed a coherent pattern.\u003c/li\u003e\n \u003cli\u003eLearning activities used in this course [lectures, small groups, demonstrations, labs, case discussions, quizzes, etc.] were effective.\u003c/li\u003e\n \u003cli\u003eThe course was intellectually challenging and stimulating.\u003c/li\u003e\n \u003cli\u003eExam questions for this course challenged me to critically think about concepts I was presented.\u003c/li\u003e\n \u003cli\u003eThe examinations were administered according to the Student Handbook.\u003c/li\u003e\n \u003cli\u003eThe course conformed to the schedule published in the syllabus.\u003c/li\u003e\n \u003cli\u003eThe amount of time I invested in this course was appropriate for the allotted credits.\u003c/li\u003e\n \u003cli\u003eRate the course overall.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eWe explored the relationships between component questions as well as those between the questions and VEA results in order to probe the question of teacher-effectiveness versus grade bias.\u003c/p\u003e\n\u003cp\u003eCourse evaluations were not normally distributed. \u0026nbsp;Therefore, Spearman correlation coefficients were calculated to explore the relationships between the median grade for the course, the mean SET, the percentage of A-grades, and VEA score. We also explored the relationship between the individual questions in the SET. \u0026nbsp;(Notwithstanding the well-known problems involved with treating data derived from Likert scales as continuous, we used the SET means because that is what is commonly used at many universities and colleges.)\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003eMean course evaluations were significantly positively related to median grades in the courses (rho = 0.33, \u003cem\u003eP\u003c/em\u003e = 0.0001). In addition, they were significantly positively correlated with the proportion of students earning final grades in the A-range (rho = 0.35, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.0001). On the other hand, the relationship between course evaluations and the corresponding sectional VEA scores was negative (rho = -0.18, \u003cem\u003eP\u003c/em\u003e = 0.027), as was the correlation between the VEA scores and the percentage of grades in the A-range (rho = -0.23, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.0043). These data are summarized in Table 2.\u003c/p\u003e\n\u003cp\u003eTable 2. Relationship of Response to \u0026quot;rate the course overall\u0026quot; to course median grade and VEA scores. (Cells contain Spearman\u0026apos;s correlation coefficient (rho), P-value, and number of courses.)\u003c/p\u003e\n\u003ctable border=\"1\" cellspacing=\"0\" cellpadding=\"0\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eOverall Evaluation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eMedian Grade\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003ePercent of A-grades\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eVEA score\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eOverall Evaluation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eMedian Score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e0.33\u003c/p\u003e\n \u003cp\u003e0.0001\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003ePercent of A-grades\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e0.35\u003c/p\u003e\n \u003cp\u003e0.0001\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e0.84\u003c/p\u003e\n \u003cp\u003e0.0001\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003eVEA score\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e-0.18\u003c/p\u003e\n \u003cp\u003e0.027\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e-0.14\u003c/p\u003e\n \u003cp\u003e0.075\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e-0.27\u003c/p\u003e\n \u003cp\u003e0.0007\u003c/p\u003e\n \u003cp\u003e152\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd width=\"20%\" valign=\"top\"\u003e\n \u003cp\u003e1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eThe relationships between median course grades and means SET score (positive) are shown in Figure 1. \u0026nbsp;Figure 2 shows the scatter plot and trend line for the negative relationship between mean course evaluation and VEA score.\u003c/p\u003e\n\u003cp\u003eNotably, responses to all component questions of the student evaluation were significantly correlated to each other. Spearman correlation coefficients ranged from 0.67 to 0.92, and for all P \u0026lt; \u0026nbsp; 0.0001. \u0026nbsp;This might suggest a mindless completion of the evaluation once a student had formed an overall opinion of the course. Responses to all questions were either unrelated to VEA scores (P \u0026gt; 0.05) or negatively correlated. \u0026nbsp; All were significantly and positively correlated with course median grade and with the proportion of the class earning grades in the A-range. \u0026nbsp;Even responses to the question \u0026ldquo;The course was intellectually challenging and stimulating\u0026rdquo; were positively correlated to course median and mean grades. In other words, courses were rated as more challenging when grades were higher, in contrast to what might be expected (in that more challenging courses might be expected to yield lower grades). Even more strikingly, student assessment of whether or not courses adhered to the published syllabus also varied with the median grade\u0026mdash;even though essentially no courses deviated at all from the syllabus and, if they had, it would have constituted a basis for grade appeals and formal complaints. Relatedly, the fact that students were more likely to rate \u0026ldquo;time invested in the course\u0026rdquo; as appropriate for the allocated credit hours if they received or anticipated high grades despite the negative correlation of this component with actual learning measured by VEA scores\u0026mdash;implying that students favored courses that offered higher grades without great expenditure or effort\u0026mdash;adds further support to grade-bias interpretations.\u0026nbsp;\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eOur study confirms the well-established grade bias to which SET is subject. \u0026nbsp;For the first time in a veterinary educational context, the negative relationship between SET and actual learning is illustrated.\u003c/p\u003e\n\u003cp\u003eA novel aspect of the study is that we were able to take advantage of the VEA, as an external and independent measure of learning to explore the relationship between SET and learning. \u0026nbsp;While the negative relationship between SET and VEA scores is not strong, it is statistically significant and clearly undermines any argument that favorable SET and high course grades both reflect superior teaching and, by implication, learning. This, in turn, argues for extreme caution in the use of SET for informing important, career-defining decisions such as faculty remuneration, promotion or tenure.\u003c/p\u003e\n\u003cp\u003eImportantly, with regard to the design of this study, instructors of the included courses had control over the content, pace, methods of assessment, and course design more generally. Hence, our study is not obviously subject to a criticism that has been made of multi-section validity studies: that wresting control of courses from instructors and placing it in the hands of study designers undermines the quality of the courses, obfuscates the relevance of any observed correspondence between evaluations and marks, and reduces the scope of any conclusions.(11)\u003c/p\u003e\n\u003cp\u003eThe consistent correlation between component questions of the SET, and their relationship to grades earned in courses, supports the contention that students seem prone to complete surveys without attention to detail, once having decided their overall opinion of the course. An extreme instance of lack of diligence in completion of course evaluations is a study by Uijtdehaage and O\u0026rsquo;Neal in which 66% of students completed assessments for a fictitious instructor and 49% did so even when a photograph of the phantom teacher was included.(19)\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe lack of a one-to-one relationship between subjects taught and VEA sections is a weakness of this study, but we believe that the availability of an external, objective measure that cannot be manipulated by the instructors or the institution provides a unique opportunity to study student evaluation of teaching and actual learning, offsetting this acknowledged deficit.\u003c/p\u003e\n\u003cp\u003eSecondly, in order to interpret the negative correlation between SET/grades and performance on the external exam as disconfirming evidence of teacher-effectiveness theories, one needs to assume that: (\u003cem\u003ei\u003c/em\u003e) student learning is an acceptable measure of teaching effectiveness (or a significant component thereof); and (\u003cem\u003eii\u003c/em\u003e) measuring performance on the external exam is a reasonable proxy for assessing learning. Proponents of teacher-effectiveness theories might well grant (\u003cem\u003ei\u003c/em\u003e) but argue, against (\u003cem\u003eii\u003c/em\u003e), that course grades are a better indicator of learning than a standardized exam. However, recent studies exploring and modeling the mechanisms relating SET to grade inflation cast some doubt against the validity of such an argument.(20)\u0026nbsp;Our study cannot settle this debate.\u0026nbsp;\u003c/p\u003e"},{"header":"Conclusion","content":"\u003cp\u003eStudent evaluations of teaching should be interpreted very cautiously; their use for evaluation of teachers or for salary improvement or promotion may be counter-productive and penalize some of the most effective teachers and provide unintended incentives for teachers to lower expectations of student mastery of material.\u003c/p\u003e\n"},{"header":"Abbreviations","content":"\u003cp\u003eIRB: \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Institutional Review Board\u003c/p\u003e\n\u003cp\u003eRUSVM:\u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Ross University School of Veterinary Medicine\u003c/p\u003e\n\u003cp\u003eSET: \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp;\u0026nbsp;Student Evaluation of Teaching\u003c/p\u003e\n\u003cp\u003eVEA: \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; \u0026nbsp; Veterinary Educational Assessment\u003c/p\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eEthics approval:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study was reviewed by the IRB of Ross University School of Veterinary Medicine and deemed to be exempt from review and from individual student consent in terms of\u0026nbsp;U.S. Federal Regulations 45 CFR 46.102.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConsent for publication:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAvailability of data\u003c/strong\u003e:\u003c/p\u003e\n\u003cp\u003eData are available from the corresponding author upon reasonable request.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors declare no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNone\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor Contributions:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe authors confirm contribution to the paper as follows: study conception and design: ROG and DRG; data acquisition: ROG; analysis of results: ROG; draft manuscript preparation: ROG and DRG. Both authors reviewed the results and approved the final version of this manuscript. Both authors agree to be accountable for all aspects of the work.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAcknowledgements:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNot applicable\u003cbr\u003e\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eMiller J, Seldin P. Changing practices in faculty evaluations: can better evaluation make a difference? https://www.aaup.org/article/changing-practices-faculty-evaluation: American Association of University Professors; 2014 [cited 2023 November 22].\u003c/li\u003e\n\u003cli\u003eBoring A. Gender biases in student evaluations of teaching. Journal of public economics. 2017;145:27-41.\u003c/li\u003e\n\u003cli\u003eFan Y, Shepherd LJ, Slavich E, Waters D, Stone M, Abel R, et al. Gender and cultural bias in student evaluations: Why representation matters. PLOS ONE. 2019;14(2):e0209749.\u003c/li\u003e\n\u003cli\u003eHamermesh DS, Parker A. Beauty in the classroom: Instructors pulchritude and putative pedagogical productivity. Economics of education review. 2005;24(4):369-76.\u003c/li\u003e\n\u003cli\u003eWolbring T, Riordan P. How beauty works. Theoretical mechanisms and two empirical applications on students\u0026apos; evaluation of teaching. Social science research. 2016;57:253-72.\u003c/li\u003e\n\u003cli\u003eSmith BP. Student ratings of teaching effectiveness: an analysis of end-of-course faculty evaluations. College student journal. 2007;41(4):788.\u003c/li\u003e\n\u003cli\u003eJoye S, Wilson JH. Professor Age and Gender Affect Student Perceptions and Grades. The journal of scholarship of teaching and learning. 2015;15(4):126-38.\u003c/li\u003e\n\u003cli\u003eFeistauer D, Richter T. Validity of students\u0026rsquo; evaluations of teaching: Biasing effects of likability and prior subject interest. Studies in educational evaluation. 2018;59:168-78.\u003c/li\u003e\n\u003cli\u003eCohen PA. Student Ratings of Instruction and Student Achievement: A Meta-analysis of Multisection Validity Studies. Review of educational research. 1981;51(3):281-309.\u003c/li\u003e\n\u003cli\u003eFeldman KA. The Association between Student Ratings of Specific Instructional Dimensions and Student Achievement: Refining and Extending the Synthesis of Data from Multisection Validity Studies. Research in higher education. 1989;30(6):583-645.\u003c/li\u003e\n\u003cli\u003eJohnson VE, Springer LeEIC, Ebook C, SpringerLink. Grade inflation: a crisis in college education. New York: Springer; 2003.\u003c/li\u003e\n\u003cli\u003eAnderson RE, Choi KS, Hair JF. Cognitive Consistency Theory and Student Evaluation of Teacher Effectiveness. The Journal of experimental education. 1975;44(2):64-70.\u003c/li\u003e\n\u003cli\u003eClayson DE, Frost TF, Sheffet MJ. Grades and the Student Evaluation of Instruction: A Test of the Reciprocity Effect. Academy of Management learning \u0026amp; education. 2006;5(1):52-65.\u003c/li\u003e\n\u003cli\u003eClayson DE. Student Evaluations of Teaching: Are They Related to What Students Learn?: A Meta-Analysis and Review of the Literature. Journal of marketing education. 2009;31(1):16-30.\u003c/li\u003e\n\u003cli\u003eBoring A, Ottoboni K, Stark P. Student Evaluations of Teaching (Mostly) Do Not Measure Teaching Effectiveness. ScienceOpen research. 2016.\u003c/li\u003e\n\u003cli\u003eUttl B, White CA, Gonzalez DW. Meta-analysis of faculty\u0026apos;s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in educational evaluation. 2017;54:22-42.\u003c/li\u003e\n\u003cli\u003eLooi JCL, Anderson KJ. Between SET and ASP: balancing the scales of student evaluation of teaching (SET) and teachers\u0026rsquo; assessments of student performance (ASP) for medical school education in psychiatry. Australasian psychiatry : bulletin of the Royal Australian and New Zealand College of Psychiatrists. 2018;26(6):659-61.\u003c/li\u003e\n\u003cli\u003eBeran TN, Donnon T, Hecker K. A review of student evaluation of teaching: applications to veterinary medical education. Journal of veterinary medical education. 2012;39(1):71.\u003c/li\u003e\n\u003cli\u003eUijtdehaage S, O\u0026apos;Neal C. A curious case of the phantom professor: mindless teaching evaluations by medical students. Med Educ. 2015;49(9):928-32.\u003c/li\u003e\n\u003cli\u003eStroebe W. Student Evaluations of Teaching Encourages Poor Teaching and Contributes to Grade Inflation: A Theoretical and Empirical Analysis. Basic and applied social psychology. 2020;42(4):276-94.\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":true,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"Student evaluation of teaching, Veterinary education, Veterinary Educational Assessment","lastPublishedDoi":"10.21203/rs.3.rs-4224772/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4224772/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003e\u003cstrong\u003eBackground: \u003c/strong\u003eStudent Evaluations of Teaching (SET) are routinely utilized in the assessment of university faculty for purposes of hiring, promotion, tenure, and merit-based salary increases. Despite their ubiquity, much evidence exists that they are biased, amongst other factors by expected and received student grades. To our knowledge, this issue has not been examined in veterinary education. Furthermore, it remains unclear whether the combination of higher grades and more favorable student evaluations might reflect enhanced learning. Our study evaluates the relationship between (A) student evaluations of courses in a veterinary curriculum, (B) grades earned in those courses, and (C) independent measures of learning in those subjects.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMethods:\u003c/strong\u003e The Veterinary Educational Assessment (VEA) is an independent, external exam in basic sciences subjects administered by the International Council for Veterinary Assessment and is taken by Ross University School of Veterinary Medicine (RUSVM) students in their fifth semester of study. It offers an external means of measuring student learning in specific subjects and relating them to course evaluations. RUSVM has three terms each year with three separate intakes of students. Course evaluations and student grades were recorded for courses from fall 2018 to summer 2022, spanning 12 cohorts of students, and 152 individual courses. Courses were aligned to the relevant section of the VEA taken by each cohort. Spearman correlation coefficients were calculated.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eResults:\u003c/strong\u003e Mean course evaluations were significantly positively correlated to median grade in the course (rho = 0.33, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.0001) and the proportion of students earning A-grades (rho = 0.35, \u003cem\u003eP\u003c/em\u003e \u0026lt; 0.0001). The relationship between course evaluation and relevant VEA score was negative (rho = -0.18, \u003cem\u003eP\u003c/em\u003e = 0.027), indicating that students judged courses favorably when higher grades were expected without necessarily learning more from those courses.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eConclusions:\u003c/strong\u003e We confirmed the well-known relationship between SET and student grades but, for the first time in veterinary medicine, describe a small but negative relationship between SET and an independent measure of learning. \u0026nbsp;SET should be interpreted with caution; their use for evaluation of teachers may have unintended consequences including reduced expectations for student achievement.\u003c/p\u003e","manuscriptTitle":"Student Evaluations of Teaching Do Not Reflect Student Learning: An Observational Study","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-05-17 16:39:21","doi":"10.21203/rs.3.rs-4224772/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2024-05-27T01:11:03+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-22T13:54:21+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2024-05-21T17:21:30+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"108259034567801425683715337868168733068","date":"2024-05-14T12:06:09+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"221564591246426457523328807534622871117","date":"2024-05-14T09:54:04+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2024-05-14T08:49:18+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2024-05-14T08:47:32+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2024-05-08T18:24:00+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2024-05-08T18:22:52+00:00","index":"","fulltext":""},{"type":"submitted","content":"BMC Medical Education","date":"2024-04-05T20:32:41+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"bmc-medical-education","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"meed","sideBox":"Learn more about [BMC Medical Education](http://bmcmededuc.biomedcentral.com/)","snPcode":"","submissionUrl":"https://www.editorialmanager.com/meed/default.aspx","title":"BMC Medical Education","twitterHandle":"BMC_series","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"em","reportingPortfolio":"BMC Series","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"8dcfc983-1fd3-4ddf-a547-b2850d0e64c0","owner":[],"postedDate":"May 17th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"published-in-journal","subjectAreas":[],"tags":[],"updatedAt":"2025-03-03T16:05:29+00:00","versionOfRecord":{"articleIdentity":"rs-4224772","link":"https://doi.org/10.1186/s12909-025-06896-3","journal":{"identity":"bmc-medical-education","isVorOnly":false,"title":"BMC Medical Education"},"publishedOn":"2025-02-26 15:57:45","publishedOnDateReadable":"February 26th, 2025"},"versionCreatedAt":"2024-05-17 16:39:21","video":"","vorDoi":"10.1186/s12909-025-06896-3","vorDoiUrl":"https://doi.org/10.1186/s12909-025-06896-3","workflowStages":[]},"version":"v1","identity":"rs-4224772","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4224772","identity":"rs-4224772","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}