Student perceptions of cardiovascular physiology module format in a preclinical medical science course

1 1 Student perceptions of cardiovascular physiology module format in a preclinical medical science 2 course 3 4 Kristen Scherrer 1¶, Spencer Sullivan1¶, Gary Beck Dallaghan1, Emily Moorefield1 5 6 1University of North Carolina School of Medicine at Chapel Hill, NC, USA 7 8 9 *Corresponding author 10 E-mail: [email protected] (EM) 11 12 13 ¶ These authors contributed equally to this work. 14 15 16 17 18 19 20 . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint NOTE: This preprint reports new research that has not been certified by peer review and should not be used to guide clinical practice. 2 21 Abstract: 22 Positive active learning outcomes require student engagement with foundational, preparatory material 23 prior to class. The current study involved thorough updating of cardiovascular physiology module 24 format. It then examined preclinical medical student perceptions, and midterm exam performance, after 25 using different preparatory module formats that were reviewed prior to participating in interactive 26 classroom sessions. Modules that were initially created in Articulate Storyline were updated in Articulate 27 Rise360 and introduced over a 3-year period. Both module styles contained the same cardiovascular 28 physiology content, but updated Rise360 modules presented content in multiple formats to capture a 29 variety of student learning preferences and divided each concept into several smaller topics to maintain 30 student attention. Although midterm exam performance remained unchanged, student evaluations 31 revealed that the updated Rise360 modules were more helpful with fewer technical issues indicating 32 that students preferred the interactive online modules to prepare for collaborative classroom exercises. 33 Students find updated preparatory modules to be more helpful and may therefore be more likely to 34 engage with them before class and ultimately lead to a more productive interactive classroom learning 35 experience. 36 Introduction: 37 Implementation of active-learning techniques has consistently shown to enhance concept retention and 38 student achievement [1,2]. One approach to active learning, the flipped classroom, requires that 39 students acquire foundational knowledge outside of class and then apply this knowledge through higher 40 order thinking and problem-solving exercises with the support of faculty and peers in class [3, 4]. Thus, 41 student preparation outside of class is essential for successful implementation of this type of active 42 learning technique [5]. Faculty-created modules are one way to provide students with foundational, 43 preparatory content and have the advantage of being customized to cover specific course objectives 44 useful for interactive classroom exercises. Students are responsible for a thorough and thoughtful . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 3 45 review of module content prior to class so that they are prepared to actively engage with faculty and 46 peers during the classroom session. 47 The current study therefore sought to examine student perceptions of preparatory module format when 48 learning basic cardiovascular physiology. An initial set of modules on six cardiovascular physiology 49 concepts was created in roughly the year 2000 using Articulate Storyline, which included audio, video, 50 and matching questions. Module topics included: membrane potential, cardiac cycle, electrocardiogram, 51 vascular hemodynamics, cardiac output, and regulation of mean arterial pressure. Each of the six 52 Storyline module videos had a run time of approximately one hour. Storyline modules were initially well 53 received; however, as time has progressed they have become technologically outdated as they are not 54 compatible with popular internet browsers and do not allow for variable speed playback. 55 Newer modules were created in Articulate Rise360 on the same six cardiovascular physiology concepts. 56 Although content remained the same, modernized modules presented content in multiple ways 57 including brief videos allowing variable speed playback, corresponding web-based text and graphics, 58 interactive images, and multiple-choice check-your-understanding questions. Updated modules also 59 divided each concept into several smaller topics. For example, cardiac cycle module topics include 60 events of the cardiac cycle, volumes and pressures of the cardiac cycle, heart sounds, and pressure 61 volume loops. Updated modules were introduced in a stepwise manner beginning in 2019, when 62 students were assigned two new Rise360 modules on traditionally more complex topics, the cardiac 63 cycle and cardiac output, and the original Storyline modules on the remaining topics. Then, beginning in 64 2020, and in 2021, students were assigned updated Rise360 modules on all six basic cardiovascular 65 physiology topics. The 2017-18 school years were included as a comparison group since all five modules 66 assigned were Storyline modules. Herein we examined student, end-of-block evaluations and discovered 67 that students found the updated Rise360 modules to be more helpful and have fewer technical issues 68 than the original Storyline modules. . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 4 69 Materials and Methods: 70 Student perceptions of module format were assessed by examining end-of-block evaluations. In the 71 evaluations, students rated modules, as well as other modes of instruction, using a 5-point Likert scale 72 ranging from “not at all helpful” to “extremely helpful”, or by selecting, “Did not use/NA to this block”. 73 Student ratings were compared across years using the Kruskal-Wallis Test with Mann-Whitney U test for 74 nonparametric data (IBM SPSS, v24). 75 In addition, narrative comments that included the word ‘module’ were extracted from open-ended 76 evaluation questions including, “Please comment about your experiences with content and 77 organization.”, and “Which methods and/or materials were most helpful? What could have been more 78 helpful? How?”. Narrative comments were inductively analyzed, and four common themes were 79 identified: organizational, technical, content, and general, which were further categorized as a ‘pro’ 80 comment or a ‘con’ comment (see Figure 1 caption for details). Three independent researchers then 81 deductively categorized themes from the student comments and discrepancies were discussed until full 82 agreement was reached. Data was anonymously collected and analyzed in aggregate. 83 84 Fig 1 Categorization of open-ended responses containing the word ‘module’ for each student cohort. 85 Data are presented as the percentage of comments in each category (y-axis) as well as the number of 86 comments in each category (numerical values within bars). Comments containing the word ‘module’ 87 were extracted and inductively categorized into four themes: technical, content, organizational, and 88 general. Technical comments discussed technical features like functionality, speed, and browser 89 compatibility. Content comments discussed the usefulness of the modules and other non-technical 90 aspects including the amount of detail in the module and organization of an individual module. 91 Organizational comments discussed the flow of content and instruction throughout the block including . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 5 92 the timing of modules and their connections with other material. General comments discussed non- 93 specific, generalized descriptions of modules. Comments were further divided into ‘pro’ and ‘con’ 94 categories based on whether the overall impression was favorable or not. Some comments included 95 more than one sentiment and were therefore classified into multiple categories. 96 97 Additionally, student performance on the cardiovascular block multiple-choice midterm exam was 98 analyzed across each year using the Kruskal-Wallis Test to assess comprehension and retention 99 differences in module format types. The general content and number of questions per topic remained 100 similar from year to year. 101 This study was approved by the UNC-Chapel Hill Office of Human Research Studies and conducted under 102 the guidelines of UNC IRB Study 22-2111. Consent was not obtained because all data were analyzed 103 anonymously. 104 Results: 105 A Kruskal-Wallis test demonstrated that student ratings of module helpfulness increased as additional 106 Rise360 modules were introduced, H(2) = 35.14, p < 0.001 (see Table 1 for descriptive statistics and 107 student ratings by year). Mann-Whitney tests were used to compare all groups and showed that module 108 helpfulness ratings in 2019 were significantly greater compared to 2017-18 (p = 0.022), and in 2020- 109 2021 they were significantly greater compared to 2019 (p = 0.006) and 2017-18 (p < 0.001). These 110 findings suggest students were more satisfied with the quality of updated Rise360 modules compared to 111 original Storyline modules. 112 Course Year 2017 - 2018 2019 2020 - 2021 . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 6 Students per group (n) 369 180 356 Module Rating; M (SD) 3.35 (1.24)* 3.62 (1.10)* 3.89 (1.09)* 113 114 Table 1 Modules were rated on their level of helpfulness using a 5-point Likert scale by each student 115 cohort, 2017-18, 2019, and 2020-21. * Statistical Significance using Kruskal-Wallis and Mann-Whitney 116 tests (p < 0.05). Abbreviations: n, sample size; M, mean; SD, standard deviation. 117 118 Analysis of student comments containing the word ‘module’ revealed an increased number of positive 119 comments in all four categories with the implementation of updated Rise360 modules (Figure 1). 120 Initially, in 2017-18 when students were assigned only Storyline modules, there was a high number of 121 comments associated with technology and the majority were negative (39 technical comments total; 37 122 con). These comments referenced the inability to access the modules in Chrome and the inability to 123 view the module at 2x speed, among other functional issues. Over the 3-year incorporation of the 124 Rise360 modules, the total number of technical comments decreased dramatically (37 con comments in 125 2017-18 down to 1 con comment in 2020-21) with mostly positive comments by 2020-21 (6 technical 126 comments total; 5 technical pro) and reference to the ease of use. 127 The overall number of comments regarding module content did not change drastically during the 128 module updating period. However, interestingly, the percentage of positive comments for module 129 content increased each year and in 2020-21, when all the modules assigned were Rise360 format, 85% 130 of module content comments (75 of 88 total) were positive even though the content presented 131 remained the same. Student comments regarding organization of content within the block including the 132 timing of modules and their relationship to other material also decreased in number over time and there 133 were many fewer negative organizational comments with the introduction of Rise360 modules. General . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 7 134 comments about the modules did not reference specific features and showed a similar trend with fewer 135 general con comments after incorporating the Rise360 modules. 136 Analysis of midterm exam performance across years failed to show any significant difference: 81.2% (SD 137 8.4) ,78.9% (SD 10.8), 80.7% (SD 9.5) for 2017-18, 2019, and 2020-21, respectively, (H(2) = 4.75, p = 138 0.093). This finding demonstrates that the newer modules did not directly influence immediate student 139 learning or retention. 140 Discussion: 141 In this study cardiovascular physiology modules were thoughtfully updated with the goal of increasing 142 student engagement and understanding of foundational cardiovascular physiology concepts. Original 143 Storyline modules presented each of six cardiovascular physiology concepts as a single video with a run 144 time of approximately one hour, a format which may be challenging or overwhelming for students to 145 process and comprehend. Updated Rise360 modules presented topics using multiple learning 146 modalities, which appeals to a variety of learning preferences and encourages a wide range of students 147 to interact with content. In addition, Rise360 modules separated each concept into smaller parts to 148 allow easier navigation, prevent distraction, and facilitate completion without interruption. 149 Our analysis of student evaluation ratings and comments revealed that Rise360 modules were more 150 helpful and had fewer technical issues than the original Storyline modules. Previous research 151 demonstrates that interactive modules increase student study time [6] which may enhance active 152 learning classroom activities. In addition to study time, increased usage may also translate into a greater 153 number of students that engage with the modules at a time when engagement is dwindling [7]. One 154 limitation of the current study; however, is that our institution’s learning management system was not 155 configured to track the number of students that opened modules or the amount of time each student 156 spent interacting with the modules. Although student evaluations suggest there was more engagement . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 8 157 with the updated modules, we are unable to directly compare student interaction between the different 158 module styles. 159 Interactive modules that provide learner control, graphics and videos, and self-assessments have shown 160 to significantly increase medical student learning of basic science concepts [8, 9]. The present study 161 confirms that students found the updated Rise360 modules containing these features to be more 162 helpful. However, regardless of the module format assigned, student performance on the cardiovascular 163 midterm exam remained unchanged. Our findings are consistent with other mixed results on the impact 164 of modules [6] and flipped classrooms [4] on promoting knowledge acquisition and retention. 165 One complicating factor in our study is that overall exam performance has declined over the past several 166 years, perhaps due in part to the stress of the COVID-19 pandemic. Pandemic-related stress has shown 167 to negatively impact mental health and exam performance in medical students [10, 11]. It is possible to 168 consider that students’ cardiovascular midterm exam scores may have also significantly declined if not 169 for the updated Rise360 modules. Additional research is needed to determine whether this may be the 170 case and to further investigate specific module features that may enhance student learning. 171 In conclusion, our findings suggest that careful consideration of preparatory material format is critical 172 for maximum student satisfaction in preparation for active learning classroom activities. Presenting 173 preparatory modules in a modern, interactive format with self-assessment improves student 174 perceptions of the materials. Students find updated preparatory modules to be more helpful and may 175 therefore be more likely to engage with them before class, which may lead to a more productive 176 interactive classroom learning experience. 177 References: 178 1. Freeman S, Eddy SL, McDonough M, Smith MK, Okoroafor N, Jordt H, Wenderoth MP. Active learning 179 increases student performance in science, engineering, and mathematics. Proc Natl Acad Sci U S A. 180 2014; https://doi.org/10.1073/pnas.1319030111 181 2. Rossi IV, de Lima JD, Sabatke B, Nunes MAF, Ramirez GE, Ramirez MI. Active learning tools improve . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 9 182 the learning outcomes, scientific attitude, and critical thinking in higher education: Experiences in an 183 online course during the COVID-19 pandemic. Biochem Mol Biol Educ. 2021; 184 https://doi.org/10.1002/bmb.21574 185 3. Street S, Gilliland KO, McNeil S, Royal K. The flipped classroom improved medical student 186 performance and satisfaction in a pre-clinical physiology course. Med Sci Educ. 2015; 187 https://doi.org/10.1007/s40670-014-0092-4. 188 4. Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in 189 medical education. Med Educ. 2017 https://doi.org/10.1111/medu.13272 190 5. Persky AM, McLaughlin JE. The Flipped Classroom - From Theory to Practice in Health Professional 191 Education. Am J Pharm Educ. 2017; https://doi.org/10.5688/ajpe816118 192 6. Bryner BS, Saddawi-Konefka D, Gest TR. 2008. The impact of interactive, computerized educational 193 modules on preclinical medical education. Anat Sci Educ. 2008; https://doi.org/10.1002/ase.55 194 7. White C, Bradley E, Martindale J, Roy P, Patel K, Yoon M, Worden MK. Why are medical students 195 'checking out' of active learning in a new curriculum? Med Educ. 2014; 196 https://doi.org/10.1111/medu.12356 197 8. Khalil MK, Nelson LD, Kibble JD. The use of self-learning modules to facilitate learning of basic science 198 concepts in an integrated medical curriculum. Anat Sci Educ. 2010; https://doi.org/10.1002/ase.177 199 9. Velan GM, Killen MT, Dziegielewski M, Kumar RK. Development and evaluation of a computer-assisted 200 learning module on glomerulonephritis for medical students. Med Teach 2002; 201 https://doi.org/10.1080/01421590220145806 202 10. Zis P, Artemiadis A, Bargiotas P, Nteveros A, Hadjigeorgiou GM. Medical studies during the COVID-19 203 pandemic: the impact of digital learning on medical students’ burnout and mental health. Int J Environ 204 Res Public Health. 2021; https://doi.org/10.3390/ijerph18010349. 205 11. Andersen S, Leon G, Patel D, Lee C, Simanton E. The Impact of COVID-19 on Academic Performance 206 and Personal Experience Among First-Year Medical Students. Med Sci Educ. 2022; 207 https://doi.org/10.1007/s40670-022-01537-6. 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint 10 224 . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint . CC-BY 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. (which was not certified by peer review) The copyright holder for this preprint this version posted May 29, 2023. ; https://doi.org/10.1101/2023.05.28.23290652doi: medRxiv preprint