Improving 7th grade students’ understanding of how science works

Improving 7th grade students’ understanding of how science works | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Improving 7th grade students’ understanding of how science works Norbert Reich, Yuedong Wang This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-4736519/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract We examined the effectiveness of the SciTrek outreach program involving two junior high teachers in the same school. SciTrek has run modules in 2nd -8th grade classrooms since 2010, with each classroom participating in two modules per academic year. For this study, students (206) in eight classrooms took part in one 7th grade module focused on conservation of mass lasting six days during the Fall, and a different module focused on cellular respiration and lasting eight days in the Winter. Students took a pre-assessment prior to the modules to determine their ability to identify a testable question, to plan and design an experimental investigation, to analyze and interpret data with computational thinking. Upon finishing both modules, students took the same assessment. Students improved significantly in all aspects, showing improvements of 10–45%. Students with lower academic performance (College Preparatory, three classrooms) improved more and showed more variability in their improvement than those with higher prior academic performance (Honors). The population included socioeconomically disadvantaged (57%) and Hispanic or Latino (64%) students. We discuss the importance of key features of the approach: the inclusion of four to six trained university volunteers in each classroom to guide small group discussions, and the active engagement of the 7th grade students in developing their own questions, designing their own experiments, data collection, and the defense of their conclusion to their peers. critical thinking science outreach middle school science outreach Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Introduction Two observations underpin the 2009 launch of the SciTrek outreach program from the Chemistry and Biochemistry Department at the University of California at Santa Barbara (UCSB): The broad consensus that science education at the K-12 as well as at the university levels needs to be improved for multiple reasons [ 1 – 8 ]. Also, in spite of the existence of creative, diverse and effective approaches to address this problem, there is room for improvement in student performance or retention at the national level [ 9 – 19 ]. What are the major, acknowledged challenges for K-12 science education in the US? While some challenges are grade-level specific (e.g., elementary students typically only receive on average less than three hours per week of science instruction [ 20 ]), most K-12 grades witness similar problems. An emphasis on subject matter content has been historically seen as problematic [ 21 ] although more recent developments such as Next Generation Science Standards (NGSS) and other inquiry-based instructional methods [ 22 ] support an increased focus on science practices and aspects of critical thinking. The focus on content is particularly prevalent in middle school and high school classes, and continues at the university level. Strong evidence now links this content focus to a second challenge, the attrition of K-12 students from STEM courses, with the result that insufficient students choose STEM related career paths which is particularly troubling for minorities [ 23 – 30 ]. In spite of diverse programs to address these problems, progress on improving science education remains unacceptably slow [ 1 – 8 ]. Subject matter test performance at the national and international levels and test performance on science and engineering practices remain problematic. In many cases, programs have provided convincing statistical analysis showing their effectiveness on student retention and performance [ 12 , 18 , 31 – 39 ]. These programs include professional development opportunities for teachers, outreach efforts, public participation in research, reform-based curricula changes, preservice teacher education among other efforts. These advances beg the question of why improvements in STEM performance and retention of students in STEM fields remains problematic in the US, which is particularly troubling for cohorts which have traditionally been underrepresented in STEM such as minorities and women [ 7 , 23 , 27 , 29 , 40 – 42 ]. While the use of standardized testing to gauge student achievement, both individually and collectively, has been criticized [ 34 – 36 , 43 , 44 ], it remains one metric which clearly shows very little evidence of improvement, both within the US as well as when comparing US students to international students [ 44 ]. This is true both for subject matter content as well as assessments which gauge understanding of scientific practices. Similarly, the extensive literature on the need for US workers with a STEM background has not been addressed by any significant increase in the total numbers of K-12 students obtaining university training in STEM-related fields. Particularly troubling is the lack of significant increases in women and minorities obtaining university STEM related training [ 23 , 27 , 29 , 40 , 41 ]. Methodology The SciTrek program has been run since 2010 and covers grades 2–8, and is now expanding into high school. This study is exclusively focused on modules run in the 7th grade. We describe the origin and organization of the SciTrek program, a description of the two 7th grade modules used for this study, and the content, administration, and analysis of the assessments. Origin and organization of SciTrek Given the strong evidence that effective programs to improve science education for K-12 students exist, why is progress on making real change so slow? The effectiveness of the SciTrek program in helping elementary students improve in their attitudes towards science [ 45 ] and secondary students improve in critical thinking as applied to science practices [this work], is compelling. The ability of the program to address the broader concerns, for example, of improving long term retention of minority and women in STEM studies, is an ongoing interest, not addressed here. Four insights form the basis of the SciTrek approach, presented here for the 7th grade modules, guided in part by evidence from various studies, and the corresponding author’s experience in working with university students at UCSB. First, obtaining personal guidance from an “expert” is essential for novices in STEM and non-STEM subjects [ 11 , 37 ]. This process is labor intensive and best done in a small group setting which is extremely challenging for both elementary and secondary teachers [ 3 , 20 , 31 , 46 , 47 ]. The small groups of K-12 students (four to six, and for this study, 7th grade students) which are mentored by the university volunteers over a period of five to eight days, represents a distinct feature of the program. While these university volunteers have not received any formal teaching training, they do receive training in the modules as well as how to ask leading questions [ 48 , 49 ] to encourage students to improve their ability to ask testable questions, experimental designs, interpretation of experimental results, and presentation of their findings. The multi-day discussions between volunteers and a small group of students are simply not possible by a single person (teacher) when 30 or more students are engaged in many different experiments. A typical comment from a participating 7th grade teacher supports this statement: “As a science teacher I have not seen another program like SciTrek. NGSS emphasize having students conduct experiments, but with over 200 students a day the reality of executing that is daunting. Being able to partner with UCSB students who guide students through the process makes it possible”, JH, Science Department Head, Anacapa Middle School, Ventura California . Because the volunteers are 60–70% female and 30–40% minority, these interactions help students consider how females and minorities can engage in science [ 50 – 56 ]. Second, to the extent possible, students need to think that what they are working on is at least in part driven by their own interests [ 57 ]. This feature requires attention and is very age- and grade-driven. Unguided inquiry has been shown to be less effective than guided inquiry [ 17 , 18 , 37 , 38 ]. However, rote learning from following a detailed script is quite ineffective in building student interest and confidence [ 58 ]. Third, the possibility, indeed the likelihood of experimental “failure” and subsequent analysis by students, needs to be allowed for and encouraged [ 44 ]. Fourth, secondary school science teachers, and certainly elementary school educators, have minimal exposure to authentic science investigations and instruction during their own education and teacher training [ 59 ]. SciTrek typically reaches 10–15 schools, 60–70 teachers and approximately 2000 K-12 students each year. The program started at the second-grade level, and has expanded into secondary classrooms. As a result of district scheduling policy, no single school has implemented the entire SciTrek program. The elementary program (grades 2–6) and the secondary program are organized similarly, with important differences. Because elementary teachers rarely have a science background, the program seeks to empower the teachers to take a leadership role over a two to three year period; this involves extensive consultation with SciTrek staff. At the secondary level, although the teachers generally have science undergraduate degrees, they often lack any meaningful experience in carrying out research of any type [ 44 ]. Moreover, they rarely have the time and resources to implement a program like SciTrek; interviews and testimonials from teachers involved with SciTrek at all levels strongly support this view. At the secondary level, SciTrek engages these teachers in the design of module content. The long-term goal is to have teachers run two SciTrek modules each year with minimal involvement of SciTrek staff, with the help of trained university students, experimental materials, and workbooks. The secondary program is further distinguished by the involvement of undergraduate and graduate students (SciTrek Research Group, SRG, coauthors of this work) who obtain research credit for assisting in all aspects of the program, including training university volunteers, module development, student recruitment, leading modules in classrooms, developing and administering assessments along with their statistical analysis and publication in peer reviewed journals. SRG members typically have at least one year of extensive module experience prior to joining and work 10–15 hours per week on the program. Many former SRG members are now enrolled in post graduate programs (e.g., Ph.D.) and training in diverse health science programs. The SciTrek program Recruitment, makeup and training of university students While this study is focused on the 7th grade program, SciTrek runs similar programs in grades 2–8, which is summarized here. University undergraduate and graduate students are recruited from all disciplines, but 95% are STEM majors or have an interest in education. Undergraduates enrolled in the School of Education Minor in Science and Mathematics Education as well as the Minor in Education program participate in SciTrek. Students enrolled in Chemistry 102 (Teaching Chemistry at the High School/Junior High level), 193 (Internship in Chemistry), 198 (Research in Science Education) and Education 129 (CalTeach) participate in SciTrek modules. UCSB has 23,349 undergraduates, 46% are STEM majors, and 2,965 are graduate students. UCSB volunteers (four to six undergraduate and graduate students per K-12 classroom) who participate in running the two modules featured in this study are provided volunteer notebooks, which contain the same activities outlined in the student notebooks in addition to supplementary materials that further explain concepts covered in the module (See Supplemental Files 1–6). Answers to the questions in the student notebook are included in the volunteer notebook. All UCSB volunteers are required to attend a module orientation lasting one to two hours, specific to each individual module (Best Bread, and Conservation of Mass), where SciTrek staff or SRG members thoroughly explain the activities, procedures, and goals of the module to volunteers. The classroom teachers attend these orientations to become familiar with the module and advise the university volunteers about their students. Volunteers are introduced to the activities covered in each module as well as informed as to what is expected of them when speaking to and interacting with the students. Volunteers are provided extensive guidance in asking students leading questions to improve the quality of the questions the 7th grade students want to experiment on (48,49), on their experimental designs, on their data analysis, and how best to present their data. The volunteer training and subsequent interactions with the students on this topic are key features of the program. All SciTrek volunteers are offered transportation to local schools in order to help run the module. Secondary teachers run the module in their classrooms with the help of the university volunteers. SciTrek staff are also present in the classroom to help and look for ways of improving the modules. Module description (Figs. 1 – 3 and Supplemental Files 1–6) The organization of the two 7th grade modules that make up the study (Best Bread and Conservation of Mass) are representative of other secondary modules (e.g., the 8th grade modules Waves and Germs). The Best Bread module aims to cover Next Generation Science Standards, MS-LS1-5 and MS-LS1-1 [ 22 ]. MS-LS-5 covers the construction of scientific explanations based on evidence for how environmental and genetic factors influence the growth of organisms. In particular, it asks students to construct a scientific explanation based on evidence obtained from (among other sources) their own experiments. MS-LS-1 performance expectations ask students to conduct an investigation to provide evidence that living things are made of cells. MS-LS1 and 5 cover cellular respiration as well as understanding how environmental changes may affect an organism’s ability to function. The module introduces students to the concept of cellular respiration and has them change environmental variables in a flask containing yeast, sugar, and water (Figs. 2 A- 2 E). Students record quantitative results by measuring the amount of gas produced via water displacement. Students participating in this module learn the fundamentals of experimental design, which include forming a hypothesis as well as only changing one variable at a time (Fig. 2 A, 2 E). The second module, Conservation of Mass, addresses NGSS MS-PS1-5 “Matter and its Interactions”. It introduces students to a series of reactions that explore whether mass is lost, gained, or conserved during various physical changes and chemical reactions. Students ultimately are taught the concept of open and closed systems, and are asked to design a closed system where a baking soda and vinegar reaction takes place without any loss of mass (Conservation of Mass Notebooks available as Supplemental Files 4–6). Both modules encourage students to explore the interactions of changing variables and the value of experiments that either don’t lead to a logical conclusion or disagree with a student’s expected outcome; small group discussions, led by the university volunteers are used to emphasize the importance of multiple trials. Due to scheduling constraints, SciTrek runs the modules either on Tuesday and Thursday or Monday, Wednesday and Fridays for six days. Teachers are provided “off day” activities focused on improving the subjects covered during the module such as graphing and mathematics if they choose to use them. To fit into the 7th grade curriculum, the Best Bread module is run in the fall and the Conservation of Mass module is run in the winter/spring. As in all SciTrek modules, students are provided equipment and workbooks to complete their activities and record data during each module. The day-to-day activities for the Best Bread module (Fig. 1 ) and for the Conservation of Mass Module (Supplemental Files 6,7) highlight diverse formats used in these and other modules. These include presentations by a “lead” (classroom Teacher, or a SRG member). More often, the activities occur at one table composed of four-six 7th grade students and one university volunteer. The students are guided by the volunteer at their table or the lead. Each student has their own notebook which has questions they need to answer related to the activities they either observe or carry out themselves. The volunteers are trained to be a resource for the students [ 48 , 49 ]. Groups of students are frequently asked to report to the entire class as to their results, or answers to questions posed in their workbooks (Fig. 1 ). At the end of the module, each student group summarizes their results by creating a poster that highlights their hypothesis and includes relevant graphs, tables, and a conclusion. The presentation to the class involves questions and response from the entire class of the presenting group. Assessments (See Supplemental File 8) The goal of the assessments is to test students’ understanding of critical thinking concepts stressed by the SciTrek 7th grade module curriculum—Experimental Design, Observation/Inference, and Testable Questions, within the context of the activities covered during the modules. While each of these concepts has multiple components, for the purpose of these two 7th grade modules Experimental design focused on experimental error analysis, and observation/inference focused on the distinction between simple observations and making logical conclusions from evidence. Testable questions focus on differentiating between what can or cannot be addressed by collecting evidence. The assessment design also coincides with the Three-Dimensional Learning concepts promoted in the Next Generation Science Standards [ 22 ]. The data collected in this study comes from assessments administered in the 2018–2019 7th grade academic school year between two participating teachers at Santa Barbara Junior High School in Santa Barbara, California. The teachers both have over fifteen years’ experience in teaching the 7th grade. The assessments consisted of seven questions, some with multiple parts, worth a total of 15 points. Two questions were not counted due to technical issues, resulting in a total of 13 points possible. Students were provided 10 minutes to complete the assessment and were informed that their score will not affect their academic grade. The same assessment was distributed twice during the school year, once prior to attending any SciTrek modules, and once after both modules have been completed. The assessments were individually coded and graded separately by SRG members who had no knowledge of the student identity, whether the assessment was a pre- or post-module assessment, and if the student was in an Honors or College Prep class. The final scores for each question and sub-question (4a, 4b, etc) were recorded and the average pre and post assessment scores for each question were generated. The odds ratio, 95% confidence intervals, and p-values were generated by McNemar’s test, using the function mcnemar.exact in R package exact 2x2. The results and analysis from each individual question are provided in Table 2 . Table 1 Summary table of participants in this study. Data includes the number of assessed students. Assignment of students is done by the school and guided by performance relative to others in the class at the discretion of the teacher. Teacher Teacher # 1 Teacher # 2 Total Students 110 96 Total in College Preparation 55 29 Total in Honors 55 67 Table 2: Students show significant improvements in all but one question in all areas of science process skills targeted in these modules. Each type of question is labeled with their corresponding questions in the table. Experimental design is labeled with red, observation/inference is labeled with blue, and testable questions are labeled with green. The assessment and rubric are provided (Supplemental File 7). Odds ratios were determined by comparing the proportion of students who had answered correctly on the pre assessment with the proportion of students who had answered correctly on the post assessment. 95% confidence intervals and p values were calculated based on the exact McNemar's test. Note, questions 4C and 5E were removed from the study due to technical problems. Questions Pre correct probability Post correct probability odds ratios 95% confidence intervals p-values 1a 0.44 0.57 4.88 (2.10, 13.36) <0.0001 1b 0.46 0.54 2.13 (1.12, 4.24) 0.0186 2 0.55 0.81 5.57 (2.95, 11.50) <0.0001 3 0.53 0.89 13.17 (5.79, 36.96) <0.0001 4a 0.91 0.97 3.20 (1.12, 11.17) 0.0266 4b 0.70 0.82 2.33 (1.31, 4.31) 0.0027 4d 0.75 0.85 2.50 (1.31, 5.03) 0.0038 5a 0.80 0.92 3.18 (1.58, 6.95) 0.0005 5b 0.54 0.77 4.00 (2.29, 7.41) <0.0001 5c 0.80 0.89 2.46 (1.26, 5.11) 0.0066 5d 0.86 0.90 1.57 (0.77, 3.32) 0.2433 6 0.70 0.77 1.66 (0.96, 2.92) 0.0815 7 0.54 0.65 1.87 (1.16, 3.10) 0.0242 Results Our study involved two experienced 7th grade teachers at Santa Barbara Junior High School, both of whom had participated with SciTrek previously. The Santa Barbara Junior High School student demographics are representative of schools in the Santa Barbara School District, with two thirds of the students being socio-economically disadvantaged, and nearly 70% minority. Students are placed in College Preparatory (CP) or Honors (H) classrooms by the school based on teachers’ recommendations and reassessed during the school year by the teacher. The SciTrek program was run the same way in CP and H classrooms. The distribution of College Preparatory and Honors students between the two teachers is summarized in Table I. Figure 4 shows that students in both CP and H classrooms improved significantly on the post assessment compared to the pre assessment. CP students improved 36% in overall performance while H students also improved, albeit only 12.8% (Fig. 4 ). However, this trend is not reflected with both Teacher 1 and Teacher 2 students (Fig. 5 and Tables 2 , 3 ). In fact, CP students with Teacher 2 showed smaller improvements (10.17%) than the H students (12.13%). This contrasts with the students in the classroom of Teacher 1, which showed 44.53% (CP) and 11.76% (H) improvements. The similar improvements with both teachers in the five H classrooms suggest this level of improvement is a reliable representation of what can be expected. In contrast, the CP students show much higher variability and teacher (or classroom)-dependent variation in performance. Regardless, the data in Figs. 4 and 5 provide compelling data showing significant improvement in student understanding of the critical thinking parameters (experimental design, distinguishing between observations and inference, and identifying testable questions, Figs. 5 and 6 ). Moreover, Fig. 5 clearly shows that students of both teachers improved. Table 3 Students in CP classrooms improved more than those in Honors classrooms, and all students show significant improvements in experimental design, distinguishing between observations and inferences, and identifying testable questions. Average scores of post and pre tests, and differences between post and pre scores with 95% confidence intervals. The percentage of improvements over the pre scores are also shown. Questions Post mean scores Pre mean scores Mean difference (post-pre) % improvement 95% confidence intervals p-values All 10.35 8.59 1.76 20.5 (1.43, 2.09) < 0.0001 All-Honors (H) 10.89 9.66 1.23 12.77 (0.92, 1.54) < 0.0001 All-College Prep (CP) 9.57 7.04 2.53 35.96 (1.89, 3.17) < 0.0001 All-Teacher 1 8.60 7.61 0.99 13.07 (0.53, 1.46) < 0.0001 All-Teacher 2 11.88 9.45 2.43 25.75 (2.01, 2.86) < 0.0001 Teacher 1 H 12.1 10.83 1.27 11.76 (0.84, 1.71) < 0.0001 Teacher 1 CP 11.65 8.06 3.59 44.53 (2.99, 4.19) < 0.0001 Teacher 2 H 9.90 8.70 1.20 12.13 (0.75, 1.65) < 0.0001 Teacher 2 CP 5.60 5.09 0.52 10.17 (-0.67, 1.70) 0.3784 Experiment Design 2.54 2.14 0.40 18.62 (0.24, 0.56) < 0.0001 Distinguishing observations from inferences 4.25 3.70 0.55 14.95 (0.36, 0.75) < 0.0001 Identifying testable questions 2.64 2.37 0.27 11.47 (0.15, 0.40) 0.0003 Figure 6 shows that the high variability of student performance (CP, Fig. 4 ) is observed with both teachers. And, that the lower variability for H students (Fig. 4 ) is reflected in the performance of these students with both teachers. Importantly the latter reflect data from five classrooms (122 students). Clearly, the high variability (Fig. 6 ) and very different improvements between teachers (T1, 3.59 out of 13, T2, 0.52 out of 13, Table 3 ) implicate teacher and student composition parameters. Interestingly, the pre-module performance of the students in T2 (CP, 5.09 mean, Fig. 6 ) is by far the lowest mean of all groups, including the pre-module performance of students in T1 (CP, 8.06 mean, Fig. 6 ). This may suggest variations in the ability of students in the CP classrooms which we are not aware of that impact both the initial performance, as well as potential for improvement resulting from participating in the SciTrek program. Table 2 provides details about student performance on individual questions (Supplemental file 7) and the three critical thinking parameters which the two modules focus on (Experimental design, observation/inference, and testable questions). Questions 1a and b, 2, 6, and 7 are focused on experimental design, questions 3, 5A-D (5E was removed due to problems with question design) are focused on observations versus inference, and questions 4A, B and D are focused on testable questions (note, 4C was removed for technical reasons). With the exception of questions 5D and 6, all improvements are statistically significant. Questions 1 and 6 address a common problem for students, that in spite of being told about controls and limiting the number of changing variables, in practice they often do not display such understanding. Question 1 asks students to identify a poorly designed experiment and to redesign this to improve it. Question 2 asks students to determine which of two experimental outcomes is better; one has much greater variability than the other. The concepts of accuracy and precision, so core to science, are challenging to convey to students without relying on statistical analysis. Encouragingly, students improved dramatically on this question (mean pre score of 0.55 and post score of 0.81 out of a possible score of 1), showing that this concept can be addressed without complex analysis. The design aspects of question 7 (picking the best basketball players), while somewhat open ended, asks students to first determine the flaw in the proposed method, followed by the more open ended second part, to propose an improved approach. Again, the overall mean improvement of 11% is significant. The distinction between observations, inferences and opinions is important for students to understand, whether presented in the context of a SciTrek science module, or more broadly. Questions 3 and 5A-D address this, and with the exception of question 5D, student performance improved significantly on these questions. Question 3, which showed a very large improvement of 68% (Table 2 ), asks students to determine what types of procedural problems could lead to a flawed experimental result. In the context of the SciTrek program, improving students’ ability to identify a flawed experimental design is arguably more important and perhaps teachable than being able to design an appropriate experiment. The latter requires both understanding as well as the imagination to create the needed experiment, whereas the former simply requires an understanding of the correct practices. It is important for students to appreciate that not all questions can be addressed simply by collecting data, a concept referred to in SciTrek documents as “testable questions” [ 60 ]. Question 4 (parts A, B and D) tries to address this by posing various situations ranging from opinions (A, preferred color) to a quantifiable question (D, are you above average in height)? The improvements for this topic, although statistically significant (Table 2 ) are relatively small, ranging from 8.8% (4A) to 17% (4D). This topic is inherently subtle, perhaps even subjective. However, understanding what is testable is clearly linked to envisioning how something could be tested. Thus, while a challenge to devise suitable questions for this topic, we remain committed to both improving the module activities focused on this, as well as the assessments that measure how students perform. Discussion Critical thinking in the context of science practices, which is the focus of our study, can be challenging to teach as well as to assess [ 8 , 14 , 34 , 37 – 39 , 46 ]. Even the idea of a validated means to assess critical thinking has generated diverse opinions [ 61 ]. Similarly, how to best teach such skills remains unclear [ 8 , 14 , 34 , 37 – 39 , 46 ]. The SciTrek 7th grade program described here attempts to address both of these, but is clearly a work in progress. Notably, there was no attempt to “teach to the test” for these skills, so improvements in our assessments of these skills should reflect real changes. The value of this program to teach the content covered by these two modules was not assessed in our study. As importantly, it is challenging (unresolved) to identify which features of this study are responsible for the observed improvements in critical thinking as applied to science practices. Thus, is the involvement of small groups led by university students over the ten to sixteen hours of in class activities most important? The other features, namely the 7th grade students’ active engagement in developing their own questions and experimental design, or the arguing from evidence, would seem to be a less distinctive feature since many programs of varying levels of inquiry emphasis, make use of such activities. Similarly, one could argue that it is the distinctive features of the module activities themselves that play an important role. This study cannot evaluate which part of the SciTrek approach is most important, which is the subject of ongoing studies in our program. Overall, the results are very encouraging, showing significant improvements in the three critical thinking parameters. Improvements in experimental design, observation/inference, and testable questions are observed and in all cases the improvement is supported by more than one question in the assessment. Rather than developing lesson plans that attempt to directly teach the students the critical thinking skills, the SciTrek program relies heavily on repeatedly showing students examples and encouraging discussions on those topics. For example, for the Best Bread module, students are presented at the start with the “big question” to be investigated, “How can we maximize the amount of carbon dioxide produced by yeast”? Students discuss alternatives to this question which may or may not be testable. Students are reminded throughout the module how their efforts relate to this testable question [ 46 ]. Observations of the 7th grade students involved in this study suggests that the active, guided participation of these students in carrying out experiments, many of which fail to generate a definitive conclusion, is important to their understanding of what is often presented as the “scientific method”. While this anecdotal information is not as compelling as the statistical evidence provided in this study, the frequent comments from these students such as “I learned that failure can be helpful” will hopefully have longer term consequences than simply memorizing the scientific method. Students’ ability to identify what can be addressed through experimentation (Testable Question, Questions 4A, B and D, Table II) is a foundational feature of science investigations [ 60 ]. For example, a long-term goal of the entire SciTrek program is to improve the understanding of K-12 students’ of when a study of any kind is done well (NGSS, MS-LS-1, 5). Significant improvement in all three questions is thus encouraging. Similarly, experimental design (Questions 1A, 1B, 2, 6 and 7) naturally follows from understanding if a question can be addressed experimentally. The ability of students to both judge an experimental design, and to propose a design, feature prominently in many aspects of NGSS literature (e.g., MS-LS-1, MS-LS-1, and MS PS-5). Students showed significant improvements in four of the five questions covering this topic. The third parameter covered in the assessment, distinguishing between observations and inferences, while challenging for students, is clearly an essential aspect of doing science. Five questions covered this topic (3, 5A-D), and all but 5D showed significant improvements. While encouraging, our results leave many unanswered questions. The scope of the program was developed in consultation with participating teachers, which was important in the decision to have two modules per grade level, which is currently done in all participating classrooms throughout the SciTrek program. The in-class format, relying on five to eight hours per module is a compromise between the teachers’ curriculum plans, the time needed for 7th grade students to carry out multiple rounds of experimentation along with demonstrations to initiate each module, and the recruitment and training of university students (ranging from four to six per classroom, or 300–500 per year for the entire program). Certainly, a critical part of this was whether students’ understanding of the critical thinking concepts can be changed with this format and level of engagement. A primary focus of this study was to address this latter point. Ongoing efforts are focused on determining the impact of student participation in multiple years of the SciTrek program, on critical thinking performance, as well as choices these students make regarding continuing in STEM courses in their secondary education. Of primary interest is whether the SciTrek approach provides a practical and cost-effective means to impact science education beyond the program at the University of California at Santa Barbara, a concern raised in the introduction regarding many effective programs. The engagement of university students is an important feature of the program, and we are currently working with schools and colleges/universities in regions (counties) outside of Santa Barbara County to determine if the program can work “at a distance”. This was in part motivated by the ongoing Covid19 epidemic, but this too remains of keen interest. Room for improvement The involvement of control groups (students not participating in SciTrek) is important for future studies, but can be challenging as many schools do not provide such an opportunity; using students in other schools in the district is possible, but has its own challenges. The development of assessments that are better aligned with the content and activities of the modules is also challenging; for example, the performance of students in the pre-module assessments on questions 4A, 5C and 5D are 80% or higher, making it problematic to identify improvements in the post-module assessments (Table 2 ). Another area requiring further work is to remove or minimize issues related to language, both in the running of the module (written notebooks, instructions from those running the module etc.), as well as in the assessments. Declarations Ethics statement: The UCSB IRB has authorized this study in accordance with the UC guidelines. Informed consent : A Consent to Participate and Consent to Publish) were obtained from the parent or legal guardians. Author Contribution NOR designed and oversaw the study, YW did the statistical analysis and both authors wrote the manuscript. Acknowledgement The following students helped collect the dataFriedl, Colburn, Lejano, Collin, Douille, Clarice, Moh, Jasper, Kim, Caleb, Mohebi, Suhyl, Gustafson, Afton, Nowrozani, Aria, Bolanos, Bronte, Fitzsimmons, Cory, Makhlouf, Frank, Mellor, Grace, Sabado, Mariell, Rostomian, Nairi, Beruman, RJ Data Availability Available upon request from Norbert Reich References Alberts, B. (2009). Redefining science education. Science, 323 (5913), 437. Atkinson, R.D., Mayo, M.J. (2010). Refueling the U.S. Innovation Economy: Fresh Approaches to Science, Technology, Engineering and Mathematics (STEM) Education. Rochester, NY: Social Science Research Network. Retrieved from https://itif.org/publications/2010/12/07/refueling-us-innovation-economy-fresh-approaches-stem-education Blank, R. K. (2013). Science Instructional Time Is Declining in Elementary Schools: What Are the Implications for Student Achievement and Closing the Gap? Science Education , 97 (6), 830–847. Desilver, D. (2017). U.S. students’ academic achievement still lags that of their peers in many other countries. Pew Research Center. Retrieved from https://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science . Foltz, L.G., Gannon, S.,and Kirschmann, S.L., (2014) Factors that contribute to the persistence of minority students in STEM fields: this exploratory study examined factors that contributed to the college persistence of minority students in STEM graduate programs at LMCU, providing nuance and texture to the existing theory and research. Plan High Educ . 42 (4), 46–59. Hossain, M.M., and Robinson, M.G. (2012). How to Motivate US Students to Pursue STEM (Science, Technology, Engineering and Mathematics) Careers. US-China education review, 2 , 442–451. Hughes, B. E. (2018). Coming out in STEM: Factors affecting retention of sexual minority STEM students. Science Advances , 4 (3), eaao6373. Kafka, A.L., Ebel, J.E., Barnett, M., Macherides-Moulis, A., Campbell, L., Gordon, S.E. (2006). Classroom Seismographs and the Challenge of Encouraging a Culture of Scientific Inquiry in K-12 Schools. Seismology Research Letters , 77 (6), 734–8. Clementson, J.J. (1991). A qualitative study of elementary teachers’ and students’ interactions with Full Option Science System: FOSS. (Doctoral Dissertation). University of Nebraska - Lincoln, Lincoln, Nebraska, United States. Foster, K.M., Bergin, K.B., McKenna, A.F., Millard, D.L., Perez, L.C., Prival, J.T., Rainey, D.Y., Sevian, H.M., VanderPutten, E.A., Hamos, J.E. (2010). Partnerships for STEM Education. Science , 329 (5994), 906–7. Freeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H., Wenderoth, M.P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, 111 (23), 8410–8415. Haywood, B. K., and Besley, J. C. (2014). Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public Understanding of Science, 23 (1), 92–106. Illingworth, S.,and Roop, H. (2015). Developing Key Skills as a Science Communicator: Case Studies of Two Scientist-Led Outreach Programmes. Geosciences, 5 (1), 2–14. Isik-Ercan, Z. (2020). ‘You have 25 kids playing around!’: learning to implement inquiry-based science learning in an urban second-grade classroom. International Journal of Science Education , 42 (3), 329–49. Koehler, B. G., Park, L. Y., & Kaplan, L. J. (1999). Science for Kids Outreach Programs: College Students Teaching Science to Elementary School Students and Their Parents. Journal of Chemical Education , 76 (11), 1505–1509. Laursen, S., Liston, C., Thiry, H.,and Graf, J. (2007). What good is a scientist in the classroom? Participant outcomes and program design features for a short-duration science outreach intervention in K-12 classrooms. CBE life sciences education, 6 (1), 49–64. Robardey, C.P., Allard, D.W., and Brown, D.M. (1994). An assessment of the effectiveness of Full Option Science System training for third-through sixth-grade teachers. Journal of Elementary Science Education , 6 (1), 17–29. Tarjan, L.M., de Nesnera, K., Hoffman, R. (2015). Authentic science investigation in the classroom. Science Scope, 39 (4), 42–48. Williams-Pierce, C.C. (2011). Five Key Ingredients for Improving Student Motivation. Research in Higher Education Journal , 11. Hoyer, K.M., and Sparks, D. (2017). Instructional Time for Third- and Eighth-Graders in Public and Private Schools: School Year 2011-12. Stats in Brief. NCES 2017-076. National Center for Education Statistics . Nowicki, B.L., Sullivan-Watts, B., Shim, M.K., Young, B., Pockalny, R. (2013). Factors Influencing Science Content Accuracy in Elementary Inquiry Science Lessons. Research in Science Education , 43 (3), 1135–54. Rodger W. Bybee (2014) NGSS and the Next Generation of Science Teachers, Journal of Science Teacher Education, 25:2, 211–221. National Center for Science and Engineering Statistics. (2019). Women, Minorities, and Persons with Disabilities in Science and Engineering. Retrieved from: https://ncses.nsf.gov/pubs/nsf19304/digest Noonan, R. (2017). STEM Jobs: 2017 Update. U.S. Department of Commerce Economics and Statistics Administration Office of the Chief Economist. Retrieved from https://www.commerce.gov/sites/default/files/migrated/reports/stem-jobs-2017-update.pdf Harackiewicz, J.M., Smith, J.L., Priniski, S.J. (2016). Interest Matters: The Importance of Promoting Interest in Education. Policy Insights from the Behavioral and Brain Sciences , 3 (2), 220–7. Painter, J. (2011). Autonomy, Competence, and Intrinsic Motivation in Science Education: A Self- Determination Theory Perspective. (Doctoral dissertation). University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States Palmer, R. T., Maramba, D. C., and Dancy, T. E. (2011). A Qualitative Investigation of Factors Promoting the Retention and Persistence of Students of Color in STEM. The Journal of Negro Education, 80 (4), 491–504. Swarat, S., Ortony, A., and Revelle, W. (2012). Activity matters: Understanding student interest in school science. Journal of Research in Science Teaching , 49 (4), 515–537. Syed, M., and Chemers, M. M. (2011). Ethnic Minorities and Women in STEM: Casting a Wide Net to Address a Persistent Social Problem: Ethnic Minorities and Women in STEM. Journal of Social Issues , 67 (3), 435–441. Yanowitz, K. L., and Vanderpool, S. S. (2004). Assessing Girls’ Reactions to Science Workshops. Journal of Science Education and Technology , 13 (3), 353–359. https://doi.org/10.1023/B:JOST.0000045462.08497.1c Ertmer, P.A., and Simons, K.D. (2006). Jumping the PBL Implementation Hurdle: Supporting the Efforts of K–12 Teachers. Interdisciplinary Journal of Problem-Based Learning. 1 (1), 40–54. Haefner, L.A., and Zembal-Saul, C. (2004). Learning by doing? Prospective elementary teachers' developing understandings of scientific inquiry and science teaching and learning. International Journal of Science Education, 26 (13), 1653–1674. Markowitz, D. G. (2004). Evaluation of the Long-Term Impact of a University High School Summer Science Program on Students’ Interest and Perceived Abilities in Science. Journal of Science Education and Technology, 13 (3), 395–407. Metz, C. J., Downes, S., and Metz, M. J. (2018). The in's and out's of science outreach: assessment of an engaging new program. Advances in physiology education, 42 (3), 487–492. National Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. (R.A. Duschl, H.A. Schweingruber, and A.W. Shouse, Eds.) Washington, DC: National Academies Press. National Research Council. (2014). Developing Assessments for the Next Generation Science Standards (J. W. Pellegrino, M. R. Wilson, J. A. Koenig, & A. S. Beatty, Eds.). Washington, DC: The National Academies Press. Nold, H. (2017). Using Critical Thinking Teaching Methods to Increase Student Success: An Action Research Project. International Journal of Teaching and Learning in Higher Education , 29 (1), 17–32. Prince, M. (2004). Does Active Learning Work? A Review of the Research. Journal of Engineering Education , 93 (3), 223–31. Quigley, C., Marshall, J.C., Deaton, C.C.M., Cook, M.P., Padilla, M. (2011). Challenges to Inquiry Teaching and Suggestions for How to Meet Them. Science Educator. 20 (1), 55–61. Buffington, C., Cerf, B., Jones, C., and Weinberg, B. A. (2016). STEM Training and Early Career Outcomes of Female and Male Graduate Students: Evidence from UMETRICS Data Linked to the 2010 Census. American Economic Review , 106 (5), 333–338. Estrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Gutiérrez, C. G., Hurtado, S., John, G. H., Matsui, J., McGee, R., Okpodu, C. M., Robinson, T. J., Summers, M. F., Werner-Washburne, M., and Zavala, M. (2016). Improving Underrepresented Minority Student Persistence in STEM. CBE life sciences education, 15 (3), es5. Hussar, B., Zhang, J., Hein, S., Wang, K., Roberts, A., Cui, J., Smith, M., Bullock Mann, F., Barmer, A., and Dilig, R. (2020). The Condition of Education 2020. National Center for Education Statistics. Ryan, T. G., Whitman, A. (2013). The Inequity and Effect of Standardized Literacy Testing for First Nations Students-An Ontario (Canadian) Outlook. Journal of Educational Thought. 46 (2), 163–181 Steenhof, N., Woods, N.N., Van Gerven, P.W.M., Mylopoulos, M. (2019). Productive failure as an instructional approach to promote future learning. Advances in Health Science Education , 24 (4), 739–49. Helsel, RT, Lambert, S., Dickerson, L., Strelich, J., Woods, V. and Feldwinn, D., (2021) Design of phenomenon based science outreach program and its effects on elementary students’ epistemological understanding of and attitudes towards science, School Science and Mathematics, 122 74–85. Blanchard, M., Southerland, S. A., Osborne, J. W., Sampson, V. D., Annetta, L. A., and Granger, E. M. (2010). Is inquiry possible in light of accountability? A quantitative comparison of the relative effectiveness of guided inquiry and verification laboratory instruction. Science Education, 94 (4): 577–616. Roehrig, G.H., and Luft, J.A. (2004). Constraints experienced by beginning secondary science teachers in implementing scientific inquiry lessons. International Journal of Science Education , 26 (1), 3–24. Soysal, Y., Science Teachers’ Challenging Questions for Encouraging Students to Think and Speak in Novel Ways, Science and Education (2022) December 14, https://doi.org/10.1007/s11191-022-00411-6 . Goodman, L and Berntson, G., [2000] The American Biology Teacher, Vol 7 pp 473–476. Kuchynka, SL, Eaton, A., Rivera, LM, [2022] Understanding and Addressing Gender-Based Inequities in STEM: Research Synthesis and Recommendations for U.S. K-12 Education, 16, 252–288. Jennifer DeWitt & Louise Archer (2015) Who Aspires to a Science Career? A comparison of survey responses from primary and secondary, International Journal of Science Education, Vol. 37, No. 13, 2170–2192. Ursula Kessels, Anke Heyder, Martin Latsch & Bettina Hannover, How gender differences in academic engagement relate to students’ gender identity, Educational Research Volume 56, 2014 220–229. Adam V. Maltese & Robert H. Tai, Eyeballs in the Fridge: Sources of early interest in Science, International Journal of Science Education, 32:5, 669–685. E. W. Jenkins & R. G. Pell, Me and the Environmental Challenges: A survey of English secondary school students’ attitudes towards the environment, International Journal of Science Education, 28:7, 765–780. Potvin, P., & Hasni, A. (2014a). Analysis of the decline in interest towards school science and Technology from grades 5 through 11. Journal of Science Education and Technology, 23(6), 784–802. Allison Master, Gender Stereotypes Influence Children’s STEM Motivation, Child Development Perspectives, Volume 15, Number 3, 2021, Pages 203–210. Christine D. Clayton, Teaching for Ownership in the Middle School Science Classroom: Towards Practical Inquiry in an Age of Accountability, Teaching for Ownership MGRJ Vo. 4 (4) 53–79. Richard E. Mayer, Rote Versus Meaningful Learning (2002) THEORY INTO PRACTICE, Volume 41, Number 4, 41:4, 226–232. Raphael, J., Tobias, S., Greenberg, R. (1999). Research Experience as a Component of Science and Mathematics Teacher Preparation. Journal of Science Teaching in Education , 10 (2), 147–58. Graves, C., and Rutherford, S. (2012). Writing a Scientific Research “Testable” Question: the first step in using online data sets for guided inquiry assignments. Journal of College Science Teaching, 41 , 46–52. An Verburgh, S. Francois, J. Elen and R. Janssen (2013) The Assessment of Critical Thinking Critically Assessed in Higher Education: A Validation Study of the CCTT and the HCTA, Education Research International Volume 2013, Article ID 198920, 13 pages. Additional Declarations No competing interests reported. Supplementary Files CompletedSupplemental392021.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-4736519","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":349041877,"identity":"90c4cfa0-eff0-461d-875e-a9448b85008e","order_by":0,"name":"Norbert Reich","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA2ElEQVRIiWNgGAWjYFACxgZmhgqGBCjvABAn4FSLpOUMaVoYGJgZ20jRYt7e3Pi5cJ5dHr/YATaJH3/uMPCz5xjg1SJz5mCz9MxtycWSsxPYJHvbnjFI9rzBr0VCIrGNmXfbgcQNtxPYpBkbDjMY3CBgi4T8Q6CWOQcS94O0MPw5zGBPUIsEI1BLA9AWaZAWNqAtEoS08CQ2S/McS06ccTux2RLoFx6JM88K8GthP/7wM0+NXWL/7OSDN4AhJsffnrwBrxYkwNgAInmIVT4KRsEoGAWjAA8AAH+HRaPpvzK7AAAAAElFTkSuQmCC","orcid":"","institution":"University of California, Santa Barbara","correspondingAuthor":true,"prefix":"","firstName":"Norbert","middleName":"","lastName":"Reich","suffix":""},{"id":349041878,"identity":"3ad9bdb6-f7f9-405b-af31-8995c7624696","order_by":1,"name":"Yuedong Wang","email":"","orcid":"","institution":"University of California","correspondingAuthor":false,"prefix":"","firstName":"Yuedong","middleName":"","lastName":"Wang","suffix":""}],"badges":[],"createdAt":"2024-07-13 22:08:21","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-4736519/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-4736519/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":65287845,"identity":"f187313e-95f3-4e2a-8d31-3b402a330f8c","added_by":"auto","created_at":"2024-09-25 16:25:12","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":236461,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eDay to day schedule of the Best Bread Module\u003c/strong\u003e. Note, the schedule for the Conservation of Mass module is presented in the supplementary materials (Supplemental File 8). The corresponding student, university volunteer and lead notebooks are also included in the Supplementary files (Supplemental Files 1-6).\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/027192b1e986b8072b3af50f.png"},{"id":65287849,"identity":"eba98b8d-50a3-4222-8764-248bc31143e6","added_by":"auto","created_at":"2024-09-25 16:25:12","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":255995,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eBest Bread Module \u003c/strong\u003e(see Supplemental Files 1-3 for corresponding complete notebooks)\u003cstrong\u003e. \u003c/strong\u003e(A)\u003cstrong\u003e \u003c/strong\u003eDay 1 module activity. Students are tasked with using their senses, excluding taste or touch, to observe the two different flasks, one containing a solution of yeast, sugar, and water, and the other containing a solution of just yeast and water. These observations become the basis of the questions they propose to investigate in the following days and activities. (B) Photograph of students running the control trial using 3 grams of yeast and 1 gram of cane sugar in 100 mL of 37 degree water on Day 2. Students are asked to take notes and observe while the trial is running. (C) Students are asked to draw their rendition of the apparatus in the notebook on Day 1. A sample drawing is shown. Students can use their drawings as a quick reference in later days. (D) The correctly assembled device is shown. (E) Students are asked to set up the experiment apparatus consisting of two beakers, one containing the yeast mixture, the other containing water, connected with plastic tubing. They are first asked to run the control trial with three grams of sugar and one gram of yeast and record the amount of carbon dioxide gas produced from the trial. The students are then asked to perform a second trial to ‘maximize’ the amount of carbon dioxide gas by changing one or more variables, from a list of potential variables.\u003c/p\u003e","description":"","filename":"floatimage1.png","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/0a2f4ed73749f7550965b068.png"},{"id":65287850,"identity":"4d357f67-53de-4603-ba1e-c469caa21ad9","added_by":"auto","created_at":"2024-09-25 16:25:12","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":746833,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eConservation of Mass Module \u003c/strong\u003e(see Supplemental Files 4-6 for corresponding complete notebooks)\u003cstrong\u003e. \u003c/strong\u003eOn Day 1, after observing the burning marshmallow activity, students are provided steel wool and asked to formulate and justify a hypothesis on whether the steel wool will gain or lose mass, or remain unchanged after being ignited. Students are asked to mix baking soda and vinegar together on Day 3 of the Conservation of Mass module. Students will be tasked with constructing a closed system around this reaction on Days 4 and 5.\u003c/p\u003e","description":"","filename":"floatimage2.png","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/25e0c21cf17355e88f2021dd.png"},{"id":65288005,"identity":"8e37975b-39b4-4a9a-b7b9-15ae3f5a18c3","added_by":"auto","created_at":"2024-09-25 16:33:12","extension":"jpeg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":122060,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eCollege Prep (CP) Students show Greater Improvements in the SciTrek Critical Thinking Assessment Compared to Honors (H) Students\u003c/strong\u003e. Values are derived from the mean of student performance, with bolded lines representing median values; the assessments had a total of 13 points possible. 1st and 3rd quartiles are represented by the extent of the colored boxes. The horizontal lines at the extreme ends of the plot represent the range of the sample, while the isolated circles represent outliers, determined by multiplying the Interquartile Range (IQR) by 1.5 above or below the 3rd and 1st Quartile, respectively. Median improvements were observed in post assessment scores compared to pre assessment scores for both CP and H students, with greater median improvement in CP students compared to H students.\u003c/p\u003e","description":"","filename":"floatimage3.jpeg","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/3d92df9a172021897f86a394.jpeg"},{"id":65287848,"identity":"ac474a23-dfb6-42f7-8aa3-1dff9b31b202","added_by":"auto","created_at":"2024-09-25 16:25:12","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":21081,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eStudents Show Improvements in the SciTrek Critical Thinking Assessment Independent of Instructor. \u003c/strong\u003eValues are derived from the mean of student performance, with bolded lines representing median values; the assessment had a total of 13 points. 1st and 3rd quartiles are represented by the extent of the colored boxes. The horizontal lines at the extreme ends of the plot represent the range of the sample, while the isolated circles represent outliers, determined by multiplying the Interquartile Range (IQR) by 1.5 above or below the 3rd and 1st Quartile, respectively. The classes of the two teachers who participated in this study both showed statistically significant improvement in post assessment scores compared to pre assessment scores.\u003c/p\u003e","description":"","filename":"floatimage4.png","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/671273b72290aedea0f11ade.png"},{"id":65287846,"identity":"66ddc873-b143-4aac-abe7-883b1c5be942","added_by":"auto","created_at":"2024-09-25 16:25:12","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":22231,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eOverall Improvements in SciTrek Critical Thinking Assessment Specific to Instructor and College Prep (CP) and Honors (H) standing. \u003c/strong\u003eValues are derived from the mean of student performance, with bolded lines representing median values (total points possible on the assessment is 13). 1st and 3rd quartiles are represented by the extent of the colored boxes. The horizontal lines at the extreme ends of the plot represent the range of the sample, while the isolated circles represent outliers, determined by 1.5 * Interquartile Range (IQR) above or below the 3rd and 1st Quartile, respectively. Median improvements were observed in all post assessment scores among CP and H students separated by teacher. Only one category, Teacher 2 CP observed a 3rd quartile post-pre assessment score \u0026lt;0.\u003c/p\u003e","description":"","filename":"floatimage5.png","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/572e93a56d4ca9085bf5bca2.png"},{"id":66270484,"identity":"6d3f9194-426a-44b3-9e90-b92dd73666f2","added_by":"auto","created_at":"2024-10-09 12:47:11","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":2158460,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/dfd150a6-4f48-4bda-9de2-1923e8f82488.pdf"},{"id":65287852,"identity":"82bbb58b-48ac-4b54-909f-ef3cdeafa11e","added_by":"auto","created_at":"2024-09-25 16:25:13","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":4979659,"visible":true,"origin":"","legend":"","description":"","filename":"CompletedSupplemental392021.docx","url":"https://assets-eu.researchsquare.com/files/rs-4736519/v1/1a7fc203deaade0affeeaeaa.docx"}],"financialInterests":"No competing interests reported.","formattedTitle":"Improving 7th grade students’ understanding of how science works","fulltext":[{"header":"Introduction","content":"\u003cp\u003eTwo observations underpin the 2009 launch of the SciTrek outreach program from the Chemistry and Biochemistry Department at the University of California at Santa Barbara (UCSB): The broad consensus that science education at the K-12 as well as at the university levels needs to be improved for multiple reasons [\u003cspan additionalcitationids=\"CR2 CR3 CR4 CR5 CR6 CR7\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Also, in spite of the existence of creative, diverse and effective approaches to address this problem, there is room for improvement in student performance or retention at the national level [\u003cspan additionalcitationids=\"CR10 CR11 CR12 CR13 CR14 CR15 CR16 CR17 CR18\" citationid=\"CR9\" class=\"CitationRef\"\u003e9\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e19\u003c/span\u003e].\u003c/p\u003e\n\u003ch3\u003eWhat are the major, acknowledged challenges for K-12 science education in the US?\u003c/h3\u003e\n\u003cp\u003eWhile some challenges are grade-level specific (e.g., elementary students typically only receive on average less than three hours per week of science instruction [\u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e20\u003c/span\u003e]), most K-12 grades witness similar problems. An emphasis on subject matter content has been historically seen as problematic [\u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e21\u003c/span\u003e] although more recent developments such as Next Generation Science Standards (NGSS) and other inquiry-based instructional methods [\u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e22\u003c/span\u003e] support an increased focus on science practices and aspects of critical thinking. The focus on content is particularly prevalent in middle school and high school classes, and continues at the university level. Strong evidence now links this content focus to a second challenge, the attrition of K-12 students from STEM courses, with the result that insufficient students choose STEM related career paths which is particularly troubling for minorities [\u003cspan additionalcitationids=\"CR24 CR25 CR26 CR27 CR28 CR29\" citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR30\" class=\"CitationRef\"\u003e30\u003c/span\u003e]. In spite of diverse programs to address these problems, progress on improving science education remains unacceptably slow [\u003cspan additionalcitationids=\"CR2 CR3 CR4 CR5 CR6 CR7\" citationid=\"CR1\" class=\"CitationRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e]. Subject matter test performance at the national and international levels and test performance on science and engineering practices remain problematic.\u003c/p\u003e \u003cp\u003eIn many cases, programs have provided convincing statistical analysis showing their effectiveness on student retention and performance [\u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e12\u003c/span\u003e, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan additionalcitationids=\"CR32 CR33 CR34 CR35 CR36 CR37 CR38\" citationid=\"CR31\" class=\"CitationRef\"\u003e31\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e]. These programs include professional development opportunities for teachers, outreach efforts, public participation in research, reform-based curricula changes, preservice teacher education among other efforts. These advances beg the question of why improvements in STEM performance and retention of students in STEM fields remains problematic in the US, which is particularly troubling for cohorts which have traditionally been underrepresented in STEM such as minorities and women [\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e7\u003c/span\u003e, \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan additionalcitationids=\"CR41\" citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR42\" class=\"CitationRef\"\u003e42\u003c/span\u003e]. While the use of standardized testing to gauge student achievement, both individually and collectively, has been criticized [\u003cspan additionalcitationids=\"CR35\" citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e36\u003c/span\u003e, \u003cspan citationid=\"CR43\" class=\"CitationRef\"\u003e43\u003c/span\u003e, \u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e], it remains one metric which clearly shows very little evidence of improvement, both within the US as well as when comparing US students to international students [\u003cspan citationid=\"CR44\" class=\"CitationRef\"\u003e44\u003c/span\u003e]. This is true both for subject matter content as well as assessments which gauge understanding of scientific practices. Similarly, the extensive literature on the need for US workers with a STEM background has not been addressed by any significant increase in the total numbers of K-12 students obtaining university training in STEM-related fields. Particularly troubling is the lack of significant increases in women and minorities obtaining university STEM related training [\u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e23\u003c/span\u003e, \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e27\u003c/span\u003e, \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e29\u003c/span\u003e, \u003cspan citationid=\"CR40\" class=\"CitationRef\"\u003e40\u003c/span\u003e, \u003cspan citationid=\"CR41\" class=\"CitationRef\"\u003e41\u003c/span\u003e].\u003c/p\u003e"},{"header":"Methodology","content":"\u003cp\u003eThe SciTrek program has been run since 2010 and covers grades 2\u0026ndash;8, and is now expanding into high school. This study is \u003cem\u003eexclusively\u003c/em\u003e focused on modules run in the 7th grade. We describe the origin and organization of the SciTrek program, a description of the two 7th grade modules used for this study, and the content, administration, and analysis of the assessments.\u003c/p\u003e\n\u003ch3\u003eOrigin and organization of SciTrek\u003c/h3\u003e\n\u003cp\u003eGiven the strong evidence that effective programs to improve science education for K-12 students exist, why is progress on making real change so slow? The effectiveness of the SciTrek program in helping elementary students improve in their attitudes towards science [\u003cspan class=\"CitationRef\"\u003e45\u003c/span\u003e] and secondary students improve in critical thinking as applied to science practices [this work], is compelling. The ability of the program to address the broader concerns, for example, of improving long term retention of minority and women in STEM studies, is an ongoing interest, not addressed here.\u003c/p\u003e\n\u003cp\u003eFour insights form the basis of the SciTrek approach, presented here for the 7th grade modules, guided in part by evidence from various studies, and the corresponding author\u0026rsquo;s experience in working with university students at UCSB. First, obtaining personal guidance from an \u0026ldquo;expert\u0026rdquo; is essential for novices in STEM and non-STEM subjects [\u003cspan class=\"CitationRef\"\u003e11\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e]. This process is labor intensive and best done in a small group setting which is extremely challenging for both elementary and secondary teachers [\u003cspan class=\"CitationRef\"\u003e3\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e20\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e31\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e46\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e47\u003c/span\u003e]. The small groups of K-12 students (four to six, and for this study, 7th grade students) which are mentored by the university volunteers over a period of five to eight days, represents a distinct feature of the program. While these university volunteers have not received any formal teaching training, they do receive training in the modules as well as how to ask leading questions [\u003cspan class=\"CitationRef\"\u003e48\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e] to encourage students to improve their ability to ask testable questions, experimental designs, interpretation of experimental results, and presentation of their findings. The multi-day discussions between volunteers and a small group of students are simply not possible by a single person (teacher) when 30 or more students are engaged in many different experiments. A typical comment from a participating 7th grade teacher supports this statement: \u0026ldquo;As a science teacher I have not seen another program like SciTrek. NGSS emphasize having students conduct experiments, but with over 200 students a day the reality of executing that is daunting. Being able to partner with UCSB students who guide students through the process makes it possible\u0026rdquo;, JH, \u003cem\u003eScience Department Head, Anacapa Middle School, Ventura California\u003c/em\u003e. Because the volunteers are 60\u0026ndash;70% female and 30\u0026ndash;40% minority, these interactions help students consider how females and minorities can engage in science [\u003cspan class=\"CitationRef\"\u003e50\u003c/span\u003e\u0026ndash;\u003cspan class=\"CitationRef\"\u003e56\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eSecond, to the extent possible, students need to think that what they are working on is at least in part driven by their own interests [\u003cspan class=\"CitationRef\"\u003e57\u003c/span\u003e]. This feature requires attention and is very age- and grade-driven. Unguided inquiry has been shown to be less effective than guided inquiry [\u003cspan class=\"CitationRef\"\u003e17\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e18\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e37\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e38\u003c/span\u003e]. However, rote learning from following a detailed script is quite ineffective in building student interest and confidence [\u003cspan class=\"CitationRef\"\u003e58\u003c/span\u003e]. Third, the possibility, indeed the likelihood of experimental \u0026ldquo;failure\u0026rdquo; and subsequent analysis by students, needs to be allowed for and encouraged [\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e]. Fourth, secondary school science teachers, and certainly elementary school educators, have minimal exposure to authentic science investigations and instruction during their own education and teacher training [\u003cspan class=\"CitationRef\"\u003e59\u003c/span\u003e].\u003c/p\u003e\n\u003cp\u003eSciTrek typically reaches 10\u0026ndash;15 schools, 60\u0026ndash;70 teachers and approximately 2000 K-12 students each year. The program started at the second-grade level, and has expanded into secondary classrooms. As a result of district scheduling policy, no single school has implemented the entire SciTrek program. The elementary program (grades 2\u0026ndash;6) and the secondary program are organized similarly, with important differences. Because elementary teachers rarely have a science background, the program seeks to empower the teachers to take a leadership role over a two to three year period; this involves extensive consultation with SciTrek staff. At the secondary level, although the teachers generally have science undergraduate degrees, they often lack any meaningful experience in carrying out research of any type [\u003cspan class=\"CitationRef\"\u003e44\u003c/span\u003e]. Moreover, they rarely have the time and resources to implement a program like SciTrek; interviews and testimonials from teachers involved with SciTrek at all levels strongly support this view. At the secondary level, SciTrek engages these teachers in the design of module content. The long-term goal is to have teachers run two SciTrek modules each year with minimal involvement of SciTrek staff, with the help of trained university students, experimental materials, and workbooks.\u003c/p\u003e\n\u003cp\u003eThe secondary program is further distinguished by the involvement of undergraduate and graduate students (SciTrek Research Group, SRG, coauthors of this work) who obtain research credit for assisting in all aspects of the program, including training university volunteers, module development, student recruitment, leading modules in classrooms, developing and administering assessments along with their statistical analysis and publication in peer reviewed journals. SRG members typically have at least one year of extensive module experience prior to joining and work 10\u0026ndash;15 hours per week on the program. Many former SRG members are now enrolled in post graduate programs (e.g., Ph.D.) and training in diverse health science programs.\u003c/p\u003e\n\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\n \u003ch2\u003eThe SciTrek program\u003c/h2\u003e\n \u003cdiv id=\"Sec6\" class=\"Section3\"\u003e\n \u003ch2\u003eRecruitment, makeup and training of university students\u003c/h2\u003e\n \u003cp\u003eWhile this study is focused on the 7th grade program, SciTrek runs similar programs in grades 2\u0026ndash;8, which is summarized here. University undergraduate and graduate students are recruited from all disciplines, but 95% are STEM majors or have an interest in education. Undergraduates enrolled in the School of Education Minor in Science and Mathematics Education as well as the Minor in Education program participate in SciTrek. Students enrolled in Chemistry 102 (Teaching Chemistry at the High School/Junior High level), 193 (Internship in Chemistry), 198 (Research in Science Education) and Education 129 (CalTeach) participate in SciTrek modules. UCSB has 23,349 undergraduates, 46% are STEM majors, and 2,965 are graduate students.\u003c/p\u003e\n \u003cp\u003eUCSB volunteers (four to six undergraduate and graduate students per K-12 classroom) who participate in running the two modules featured in this study are provided volunteer notebooks, which contain the same activities outlined in the student notebooks in addition to supplementary materials that further explain concepts covered in the module (See Supplemental Files 1\u0026ndash;6). Answers to the questions in the student notebook are included in the volunteer notebook. All UCSB volunteers are required to attend a module orientation lasting one to two hours, specific to each individual module (Best Bread, and Conservation of Mass), where SciTrek staff or SRG members thoroughly explain the activities, procedures, and goals of the module to volunteers. The classroom teachers attend these orientations to become familiar with the module and advise the university volunteers about their students. Volunteers are introduced to the activities covered in each module as well as informed as to what is expected of them when speaking to and interacting with the students. Volunteers are provided extensive guidance in asking students leading questions to improve the quality of the questions the 7th grade students want to experiment on (48,49), on their experimental designs, on their data analysis, and how best to present their data. The volunteer training and subsequent interactions with the students on this topic are key features of the program. All SciTrek volunteers are offered transportation to local schools in order to help run the module. Secondary teachers run the module in their classrooms with the help of the university volunteers. SciTrek staff are also present in the classroom to help and look for ways of improving the modules.\u003c/p\u003e\n \u003c/div\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec7\" class=\"Section2\"\u003e\n \u003ch2\u003eModule description (Figs. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e\u0026ndash;\u003cspan class=\"InternalRef\"\u003e3\u003c/span\u003e and Supplemental Files 1\u0026ndash;6)\u003c/h2\u003e\n \u003cp\u003eThe organization of the two 7th grade modules that make up the study (Best Bread and Conservation of Mass) are representative of other secondary modules (e.g., the 8th grade modules Waves and Germs). The Best Bread module aims to cover Next Generation Science Standards, MS-LS1-5 and MS-LS1-1 [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e]. MS-LS-5 covers the construction of scientific explanations based on evidence for how environmental and genetic factors influence the growth of organisms. In particular, it asks students to construct a scientific explanation based on evidence obtained from (among other sources) their own experiments. MS-LS-1 performance expectations ask students to conduct an investigation to provide evidence that living things are made of cells. MS-LS1 and 5 cover cellular respiration as well as understanding how environmental changes may affect an organism\u0026rsquo;s ability to function. The module introduces students to the concept of cellular respiration and has them change environmental variables in a flask containing yeast, sugar, and water (Figs. \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003eA- \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003eE). Students record quantitative results by measuring the amount of gas produced via water displacement. Students participating in this module learn the fundamentals of experimental design, which include forming a hypothesis as well as only changing one variable at a time (Fig. \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003eA, \u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003eE). The second module, Conservation of Mass, addresses NGSS MS-PS1-5 \u0026ldquo;Matter and its Interactions\u0026rdquo;. It introduces students to a series of reactions that explore whether mass is lost, gained, or conserved during various physical changes and chemical reactions. Students ultimately are taught the concept of open and closed systems, and are asked to design a closed system where a baking soda and vinegar reaction takes place without any loss of mass (Conservation of Mass Notebooks available as Supplemental Files 4\u0026ndash;6). Both modules encourage students to explore the interactions of changing variables and the value of experiments that either don\u0026rsquo;t lead to a logical conclusion or disagree with a student\u0026rsquo;s expected outcome; small group discussions, led by the university volunteers are used to emphasize the importance of multiple trials.\u003c/p\u003e\n \u003cp\u003eDue to scheduling constraints, SciTrek runs the modules either on Tuesday and Thursday or Monday, Wednesday and Fridays for six days. Teachers are provided \u0026ldquo;off day\u0026rdquo; activities focused on improving the subjects covered during the module such as graphing and mathematics if they choose to use them. To fit into the 7th grade curriculum, the Best Bread module is run in the fall and the Conservation of Mass module is run in the winter/spring. As in all SciTrek modules, students are provided equipment and workbooks to complete their activities and record data during each module.\u003c/p\u003e\n \u003cp\u003eThe day-to-day activities for the Best Bread module (Fig. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e) and for the Conservation of Mass Module (Supplemental Files 6,7) highlight diverse formats used in these and other modules. These include presentations by a \u0026ldquo;lead\u0026rdquo; (classroom Teacher, or a SRG member). More often, the activities occur at one table composed of four-six 7th grade students and one university volunteer. The students are guided by the volunteer at their table or the lead. Each student has their own notebook which has questions they need to answer related to the activities they either observe or carry out themselves. The volunteers are trained to be a resource for the students [\u003cspan class=\"CitationRef\"\u003e48\u003c/span\u003e, \u003cspan class=\"CitationRef\"\u003e49\u003c/span\u003e]. Groups of students are frequently asked to report to the entire class as to their results, or answers to questions posed in their workbooks (Fig. \u003cspan class=\"InternalRef\"\u003e1\u003c/span\u003e). At the end of the module, each student group summarizes their results by creating a poster that highlights their hypothesis and includes relevant graphs, tables, and a conclusion. The presentation to the class involves questions and response from the entire class of the presenting group.\u003c/p\u003e\n\u003c/div\u003e\n\u003cdiv id=\"Sec8\" class=\"Section2\"\u003e\n \u003ch2\u003eAssessments (See Supplemental File 8)\u003c/h2\u003e\n \u003cp\u003eThe goal of the assessments is to test students\u0026rsquo; understanding of critical thinking concepts stressed by the SciTrek 7th grade module curriculum\u0026mdash;Experimental Design, Observation/Inference, and Testable Questions, within the context of the activities covered during the modules. While each of these concepts has multiple components, for the purpose of these two 7th grade modules Experimental design focused on experimental error analysis, and observation/inference focused on the distinction between simple observations and making logical conclusions from evidence. Testable questions focus on differentiating between what can or cannot be addressed by collecting evidence. The assessment design also coincides with the Three-Dimensional Learning concepts promoted in the Next Generation Science Standards [\u003cspan class=\"CitationRef\"\u003e22\u003c/span\u003e].\u003c/p\u003e\n \u003cp\u003eThe data collected in this study comes from assessments administered in the 2018\u0026ndash;2019 7th grade academic school year between two participating teachers at Santa Barbara Junior High School in Santa Barbara, California. The teachers both have over fifteen years\u0026rsquo; experience in teaching the 7th grade. The assessments consisted of seven questions, some with multiple parts, worth a total of 15 points. Two questions were not counted due to technical issues, resulting in a total of 13 points possible. Students were provided 10 minutes to complete the assessment and were informed that their score will not affect their academic grade. The same assessment was distributed twice during the school year, once prior to attending any SciTrek modules, and once after both modules have been completed.\u003c/p\u003e\n \u003cp\u003eThe assessments were individually coded and graded separately by SRG members who had no knowledge of the student identity, whether the assessment was a pre- or post-module assessment, and if the student was in an Honors or College Prep class. The final scores for each question and sub-question (4a, 4b, etc) were recorded and the average pre and post assessment scores for each question were generated. The odds ratio, 95% confidence intervals, and p-values were generated by McNemar\u0026rsquo;s test, using the function mcnemar.exact in R package exact 2x2. The results and analysis from each individual question are provided in Table\u0026nbsp;\u003cspan class=\"InternalRef\"\u003e2\u003c/span\u003e.\u003c/p\u003e\n \u003cdiv class=\"gridtable\"\u003e\u0026nbsp;\u003ctable id=\"Tab1\" border=\"1\"\u003e\n \u003ccaption language=\"En\"\u003e\n \u003cdiv class=\"CaptionNumber\"\u003eTable 1\u003c/div\u003e\n \u003cdiv class=\"CaptionContent\"\u003e\n \u003cp\u003e\u003cstrong\u003eSummary table of participants in this study.\u003c/strong\u003e Data includes the number of assessed students. Assignment of students is done by the school and guided by performance relative to others in the class at the discretion of the teacher.\u003c/p\u003e\n \u003c/div\u003e\n \u003c/caption\u003e\n \u003ccolgroup cols=\"3\"\u003e\u003c/colgroup\u003e\n \u003cthead\u003e\n \u003ctr\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTeacher\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTeacher # 1\u003c/p\u003e\n \u003c/th\u003e\n \u003cth align=\"left\"\u003e\n \u003cp\u003eTeacher # 2\u003c/p\u003e\n \u003c/th\u003e\n \u003c/tr\u003e\n \u003c/thead\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTotal Students\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e110\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e96\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTotal in College Preparation\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e29\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd align=\"left\"\u003e\n \u003cp\u003eTotal in Honors\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e55\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd align=\"char\"\u003e\n \u003cp\u003e67\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n \u003c/div\u003e\n \u003cp\u003e\u003cstrong\u003eTable 2: Students show significant improvements in all but one question in all areas of science process skills targeted in these modules.\u0026nbsp;\u003c/strong\u003eEach type of question is labeled with their corresponding questions in the table. Experimental design is labeled with red, observation/inference is labeled with blue, and testable questions are labeled with green. The assessment and rubric are provided (Supplemental File 7). Odds ratios were determined by comparing the proportion of students who had answered correctly on the pre assessment with the proportion of students who had answered correctly on the post assessment. 95% confidence intervals and p values were calculated based on the exact McNemar\u0026apos;s test. Note, questions 4C and 5E were removed from the study due to technical problems.\u0026nbsp;\u003c/p\u003e\n \u003ctable style=\"border-collapse: collapse;border: none;width: 622px;\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003eQuestions\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: 1pt solid black;border-right: 1pt solid black;border-bottom: 1pt solid black;border-image: initial;border-left: none;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003ePre correct probability\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: 1pt solid black;border-right: 1pt solid black;border-bottom: 1pt solid black;border-image: initial;border-left: none;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003ePost correct probability\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: 1pt solid black;border-right: 1pt solid black;border-bottom: 1pt solid black;border-image: initial;border-left: none;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003eodds ratios\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: 1pt solid black;border-right: 1pt solid black;border-bottom: 1pt solid black;border-image: initial;border-left: none;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e95% confidence intervals\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: 1pt solid black;border-right: 1pt solid black;border-bottom: 1pt solid black;border-image: initial;border-left: none;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003ep-values\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(221, 126, 107);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e1a\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.44\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.57\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e4.88\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(2.10, 13.36)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e\u0026lt;0.0001\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(221, 126, 107);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e1b\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.46\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.54\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e2.13\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.12, 4.24)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0186\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(221, 126, 107);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e2\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.55\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.81\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e5.57\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(2.95, 11.50)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e\u0026lt;0.0001\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(164, 194, 244);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e3\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.53\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.89\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e13.17\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(5.79, 36.96)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e\u0026lt;0.0001\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(182, 215, 168);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e4a\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.91\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.97\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e3.20\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.12, 11.17)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0266\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(182, 215, 168);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e4b\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.70\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.82\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e2.33\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.31, 4.31)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0027\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(182, 215, 168);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e4d\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.75\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.85\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e2.50\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.31, 5.03)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0038\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(164, 194, 244);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e5a\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.80\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.92\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e3.18\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.58, 6.95)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0005\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(164, 194, 244);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e5b\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.54\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.77\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e4.00\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(2.29, 7.41)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e\u0026lt;0.0001\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(164, 194, 244);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e5c\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.80\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.89\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e2.46\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.26, 5.11)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0066\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(164, 194, 244);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e5d\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.86\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.90\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e1.57\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(0.77, 3.32)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.2433\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(221, 126, 107);padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e6\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.70\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.77\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e1.66\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(0.96, 2.92)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;height: 14.25pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0815\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 63.75pt;border-right: 1pt solid black;border-bottom: 1pt solid black;border-left: 1pt solid black;border-image: initial;border-top: none;background: rgb(221, 126, 107);padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;color:black;\"\u003e7\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 73.5pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.54\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 81pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.65\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 74pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e1.87\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 102.25pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e(1.16, 3.10)\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 72.1pt;border-top: none;border-left: none;border-bottom: 1pt solid black;border-right: 1pt solid black;padding: 0cm 5.4pt;vertical-align: top;\"\u003e\n \u003cp style='margin:0cm;font-size:16px;font-family:\"Times New Roman\",serif;text-align:center;'\u003e\u003cspan style=\"font-family:Cambria;\"\u003e0.0242\u003c/span\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n \u003c/table\u003e\n\u003c/div\u003e"},{"header":"Results","content":"\u003cp\u003e Our study involved two experienced 7th grade teachers at Santa Barbara Junior High School, both of whom had participated with SciTrek previously. The Santa Barbara Junior High School student demographics are representative of schools in the Santa Barbara School District, with two thirds of the students being socio-economically disadvantaged, and nearly 70% minority. Students are placed in College Preparatory (CP) or Honors (H) classrooms by the school based on teachers\u0026rsquo; recommendations and reassessed during the school year by the teacher. The SciTrek program was run the same way in CP and H classrooms. The distribution of College Preparatory and Honors students between the two teachers is summarized in Table I.\u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e shows that students in both CP and H classrooms improved significantly on the post assessment compared to the pre assessment. CP students improved 36% in overall performance while H students also improved, albeit only 12.8% (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e). However, this trend is not reflected with both Teacher 1 and Teacher 2 students (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e and Tables\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e,\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e). In fact, CP students with Teacher 2 showed smaller improvements (10.17%) than the H students (12.13%). This contrasts with the students in the classroom of Teacher 1, which showed 44.53% (CP) and 11.76% (H) improvements. The similar improvements with both teachers in the five H classrooms suggest this level of improvement is a reliable representation of what can be expected. In contrast, the CP students show much higher variability and teacher (or classroom)-dependent variation in performance. Regardless, the data in Figs.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e and \u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e provide compelling data showing significant improvement in student understanding of the critical thinking parameters (experimental design, distinguishing between observations and inference, and identifying testable questions, Figs.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e and \u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e). Moreover, Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e5\u003c/span\u003e clearly shows that students of both teachers improved.\u003c/p\u003e \u003cp\u003e \u003cdiv class=\"gridtable\"\u003e\u003ctable float=\"Yes\" id=\"Tab3\" border=\"1\"\u003e \u003ccaption language=\"En\"\u003e \u003cdiv class=\"CaptionNumber\"\u003eTable 3\u003c/div\u003e \u003cdiv class=\"CaptionContent\"\u003e \u003cp\u003e\u003cb\u003eStudents in CP classrooms improved more than those in Honors classrooms, and all students show significant improvements in experimental design, distinguishing between observations and inferences, and identifying testable questions.\u003c/b\u003e Average scores of post and pre tests, and differences between post and pre scores with 95% confidence intervals. The percentage of improvements over the pre scores are also shown.\u003c/p\u003e \u003c/div\u003e \u003c/caption\u003e \u003ccolgroup cols=\"7\"\u003e \u003cdiv align=\"left\" class=\"colspec\" colname=\"c1\" colnum=\"1\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c2\" colnum=\"2\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c3\" colnum=\"3\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c4\" colnum=\"4\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c5\" colnum=\"5\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c6\" colnum=\"6\"\u003e\u003c/div\u003e \u003cdiv align=\"char\" char=\".\" class=\"colspec\" colname=\"c7\" colnum=\"7\"\u003e\u003c/div\u003e \u003cthead\u003e \u003ctr\u003e \u003cth align=\"left\" colname=\"c1\"\u003e \u003cp\u003eQuestions\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c2\"\u003e \u003cp\u003ePost mean scores\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c3\"\u003e \u003cp\u003ePre mean scores\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c4\"\u003e \u003cp\u003eMean difference\u003c/p\u003e \u003cp\u003e(post-pre)\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c5\"\u003e \u003cp\u003e% improvement\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c6\"\u003e \u003cp\u003e95% confidence intervals\u003c/p\u003e \u003c/th\u003e \u003cth align=\"left\" colname=\"c7\"\u003e \u003cp\u003ep-values\u003c/p\u003e \u003c/th\u003e \u003c/tr\u003e \u003c/thead\u003e \u003ctbody\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.35\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8.59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e20.5\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(1.43, 2.09)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll-Honors (H)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e10.89\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9.66\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.23\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e12.77\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.92, 1.54)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll-College Prep (CP)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e9.57\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7.04\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e35.96\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(1.89, 3.17)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll-Teacher 1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e8.60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e7.61\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.99\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e13.07\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.53, 1.46)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eAll-Teacher 2\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e11.88\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e9.45\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e2.43\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e25.75\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(2.01, 2.86)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTeacher 1 H\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e12.1\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e10.83\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e11.76\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.84, 1.71)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTeacher 1 CP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e11.65\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8.06\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e3.59\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e44.53\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(2.99, 4.19)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTeacher 2 H\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e9.90\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e8.70\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e1.20\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e12.13\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.75, 1.65)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eTeacher 2 CP\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e5.60\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e5.09\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.52\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e10.17\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(-0.67, 1.70)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.3784\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eExperiment Design\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2.54\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2.14\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.40\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e18.62\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.24, 0.56)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eDistinguishing observations from inferences\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e4.25\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e3.70\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.55\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e14.95\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.36, 0.75)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e\u0026lt;\u0026thinsp;0.0001\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003ctr\u003e \u003ctd align=\"left\" colname=\"c1\"\u003e \u003cp\u003eIdentifying testable questions\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c2\"\u003e \u003cp\u003e2.64\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c3\"\u003e \u003cp\u003e2.37\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c4\"\u003e \u003cp\u003e0.27\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c5\"\u003e \u003cp\u003e11.47\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c6\"\u003e \u003cp\u003e(0.15, 0.40)\u003c/p\u003e \u003c/td\u003e \u003ctd align=\"char\" char=\".\" colname=\"c7\"\u003e \u003cp\u003e0.0003\u003c/p\u003e \u003c/td\u003e \u003c/tr\u003e \u003c/tbody\u003e \u003c/colgroup\u003e \u003c/table\u003e\u003c/div\u003e \u003c/p\u003e \u003cp\u003eFigure \u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e shows that the high variability of student performance (CP, Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e) is observed with both teachers. And, that the lower variability for H students (Fig.\u0026nbsp;\u003cspan refid=\"Fig4\" class=\"InternalRef\"\u003e4\u003c/span\u003e) is reflected in the performance of these students with both teachers. Importantly the latter reflect data from five classrooms (122 students). Clearly, the high variability (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e) and very different improvements between teachers (T1, 3.59 out of 13, T2, 0.52 out of 13, Table\u0026nbsp;\u003cspan refid=\"Tab3\" class=\"InternalRef\"\u003e3\u003c/span\u003e) implicate teacher and student composition parameters. Interestingly, the pre-module performance of the students in T2 (CP, 5.09 mean, Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e) is by far the lowest mean of all groups, including the pre-module performance of students in T1 (CP, 8.06 mean, Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e6\u003c/span\u003e). This may suggest variations in the ability of students in the CP classrooms which we are not aware of that impact both the initial performance, as well as potential for improvement resulting from participating in the SciTrek program.\u003c/p\u003e \u003cp\u003eTable\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e provides details about student performance on individual questions (Supplemental file 7) and the three critical thinking parameters which the two modules focus on (Experimental design, observation/inference, and testable questions). Questions 1a and b, 2, 6, and 7 are focused on experimental design, questions 3, 5A-D (5E was removed due to problems with question design) are focused on observations versus inference, and questions 4A, B and D are focused on testable questions (note, 4C was removed for technical reasons). With the exception of questions 5D and 6, all improvements are statistically significant. Questions 1 and 6 address a common problem for students, that in spite of being told about controls and limiting the number of changing variables, in practice they often do not display such understanding. Question 1 asks students to identify a poorly designed experiment and to redesign this to improve it. Question 2 asks students to determine which of two experimental outcomes is better; one has much greater variability than the other. The concepts of accuracy and precision, so core to science, are challenging to convey to students without relying on statistical analysis. Encouragingly, students improved dramatically on this question (mean pre score of 0.55 and post score of 0.81 out of a possible score of 1), showing that this concept can be addressed without complex analysis. The design aspects of question 7 (picking the best basketball players), while somewhat open ended, asks students to first determine the flaw in the proposed method, followed by the more open ended second part, to propose an improved approach. Again, the overall mean improvement of 11% is significant.\u003c/p\u003e \u003cp\u003eThe distinction between observations, inferences and opinions is important for students to understand, whether presented in the context of a SciTrek science module, or more broadly. Questions 3 and 5A-D address this, and with the exception of question 5D, student performance improved significantly on these questions. Question 3, which showed a very large improvement of 68% (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e), asks students to determine what types of procedural problems could lead to a flawed experimental result. In the context of the SciTrek program, improving students\u0026rsquo; ability to identify a flawed experimental design is arguably more important and perhaps teachable than being able to design an appropriate experiment. The latter requires both understanding as well as the imagination to create the needed experiment, whereas the former simply requires an understanding of the correct practices.\u003c/p\u003e \u003cp\u003eIt is important for students to appreciate that not all questions can be addressed simply by collecting data, a concept referred to in SciTrek documents as \u0026ldquo;testable questions\u0026rdquo; [\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e]. Question 4 (parts A, B and D) tries to address this by posing various situations ranging from opinions (A, preferred color) to a quantifiable question (D, are you above average in height)? The improvements for this topic, although statistically significant (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e) are relatively small, ranging from 8.8% (4A) to 17% (4D). This topic is inherently subtle, perhaps even subjective. However, understanding what is testable is clearly linked to envisioning how something could be tested. Thus, while a challenge to devise suitable questions for this topic, we remain committed to both improving the module activities focused on this, as well as the assessments that measure how students perform.\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eCritical thinking in the context of science practices, which is the focus of our study, can be challenging to teach as well as to assess [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan additionalcitationids=\"CR38\" citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e, \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e]. Even the idea of a validated means to assess critical thinking has generated diverse opinions [\u003cspan citationid=\"CR61\" class=\"CitationRef\"\u003e61\u003c/span\u003e]. Similarly, how to best teach such skills remains unclear [\u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e8\u003c/span\u003e, \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e14\u003c/span\u003e, \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e34\u003c/span\u003e, \u003cspan additionalcitationids=\"CR38\" citationid=\"CR37\" class=\"CitationRef\"\u003e37\u003c/span\u003e\u0026ndash;\u003cspan citationid=\"CR39\" class=\"CitationRef\"\u003e39\u003c/span\u003e, \u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e]. The SciTrek 7th grade program described here attempts to address both of these, but is clearly a work in progress. Notably, there was no attempt to \u0026ldquo;teach to the test\u0026rdquo; for these skills, so improvements in our assessments of these skills should reflect real changes. The value of this program to teach the content covered by these two modules was not assessed in our study.\u003c/p\u003e \u003cp\u003eAs importantly, it is challenging (unresolved) to identify which features of this study are responsible for the observed improvements in critical thinking as applied to science practices. Thus, is the involvement of small groups led by university students over the ten to sixteen hours of in class activities most important? The other features, namely the 7th grade students\u0026rsquo; active engagement in developing their own questions and experimental design, or the arguing from evidence, would seem to be a less distinctive feature since many programs of varying levels of inquiry emphasis, make use of such activities. Similarly, one could argue that it is the distinctive features of the module activities themselves that play an important role. This study cannot evaluate which part of the SciTrek approach is most important, which is the subject of ongoing studies in our program.\u003c/p\u003e \u003cp\u003eOverall, the results are very encouraging, showing significant improvements in the three critical thinking parameters. Improvements in experimental design, observation/inference, and testable questions are observed and in all cases the improvement is supported by more than one question in the assessment. Rather than developing lesson plans that attempt to directly teach the students the critical thinking skills, the SciTrek program relies heavily on repeatedly showing students examples and encouraging discussions on those topics. For example, for the Best Bread module, students are presented at the start with the \u0026ldquo;big question\u0026rdquo; to be investigated, \u0026ldquo;How can we maximize the amount of carbon dioxide produced by yeast\u0026rdquo;? Students discuss alternatives to this question which may or may not be testable. Students are reminded throughout the module how their efforts relate to this testable question [\u003cspan citationid=\"CR46\" class=\"CitationRef\"\u003e46\u003c/span\u003e].\u003c/p\u003e \u003cp\u003eObservations of the 7th grade students involved in this study suggests that the active, guided participation of these students in carrying out experiments, many of which fail to generate a definitive conclusion, is important to their understanding of what is often presented as the \u0026ldquo;scientific method\u0026rdquo;. While this anecdotal information is not as compelling as the statistical evidence provided in this study, the frequent comments from these students such as \u0026ldquo;I learned that failure can be helpful\u0026rdquo; will hopefully have longer term consequences than simply memorizing the scientific method.\u003c/p\u003e \u003cp\u003eStudents\u0026rsquo; ability to identify what can be addressed through experimentation (Testable Question, Questions 4A, B and D, Table II) is a foundational feature of science investigations [\u003cspan citationid=\"CR60\" class=\"CitationRef\"\u003e60\u003c/span\u003e]. For example, a long-term goal of the entire SciTrek program is to improve the understanding of K-12 students\u0026rsquo; of when a study of any kind is done well (NGSS, MS-LS-1, 5). Significant improvement in all three questions is thus encouraging. Similarly, experimental design (Questions 1A, 1B, 2, 6 and 7) naturally follows from understanding if a question can be addressed experimentally. The ability of students to both judge an experimental design, and to propose a design, feature prominently in many aspects of NGSS literature (e.g., MS-LS-1, MS-LS-1, and MS PS-5). Students showed significant improvements in four of the five questions covering this topic. The third parameter covered in the assessment, distinguishing between observations and inferences, while challenging for students, is clearly an essential aspect of doing science. Five questions covered this topic (3, 5A-D), and all but 5D showed significant improvements.\u003c/p\u003e \u003cp\u003eWhile encouraging, our results leave many unanswered questions. The scope of the program was developed in consultation with participating teachers, which was important in the decision to have two modules per grade level, which is currently done in all participating classrooms throughout the SciTrek program. The in-class format, relying on five to eight hours per module is a compromise between the teachers\u0026rsquo; curriculum plans, the time needed for 7th grade students to carry out multiple rounds of experimentation along with demonstrations to initiate each module, and the recruitment and training of university students (ranging from four to six per classroom, or 300\u0026ndash;500 per year for the entire program). Certainly, a critical part of this was whether students\u0026rsquo; understanding of the critical thinking concepts can be changed with this format and level of engagement. A primary focus of this study was to address this latter point. Ongoing efforts are focused on determining the impact of student participation in multiple years of the SciTrek program, on critical thinking performance, as well as choices these students make regarding continuing in STEM courses in their secondary education.\u003c/p\u003e \u003cp\u003eOf primary interest is whether the SciTrek approach provides a practical and cost-effective means to impact science education beyond the program at the University of California at Santa Barbara, a concern raised in the introduction regarding many effective programs. The engagement of university students is an important feature of the program, and we are currently working with schools and colleges/universities in regions (counties) outside of Santa Barbara County to determine if the program can work \u0026ldquo;at a distance\u0026rdquo;. This was in part motivated by the ongoing Covid19 epidemic, but this too remains of keen interest.\u003c/p\u003e \u003cdiv id=\"Sec11\" class=\"Section2\"\u003e \u003ch2\u003eRoom for improvement\u003c/h2\u003e \u003cp\u003eThe involvement of control groups (students not participating in SciTrek) is important for future studies, but can be challenging as many schools do not provide such an opportunity; using students in other schools in the district is possible, but has its own challenges. The development of assessments that are better aligned with the content and activities of the modules is also challenging; for example, the performance of students in the pre-module assessments on questions 4A, 5C and 5D are 80% or higher, making it problematic to identify improvements in the post-module assessments (Table\u0026nbsp;\u003cspan refid=\"Tab2\" class=\"InternalRef\"\u003e2\u003c/span\u003e). Another area requiring further work is to remove or minimize issues related to language, both in the running of the module (written notebooks, instructions from those running the module etc.), as well as in the assessments.\u003c/p\u003e \u003c/div\u003e"},{"header":"Declarations","content":"\u003ch2\u003eEthics statement:\u003c/h2\u003e\n\u003cp\u003eThe UCSB IRB has authorized this study in accordance with the UC guidelines.\u003c/p\u003e\u003cp\u003e \u003ch2\u003e Informed consent\u003c/b\u003e:\u003c/h2\u003e \u003cp\u003e A Consent to Participate and Consent to Publish) were obtained from the parent or legal guardians.\u003c/p\u003e \u003c/p\u003e\u003ch2\u003eAuthor Contribution\u003c/h2\u003e\u003cp\u003eNOR designed and oversaw the study, YW did the statistical analysis and both authors wrote the manuscript.\u003c/p\u003e\u003ch2\u003eAcknowledgement\u003c/h2\u003e\u003cp\u003eThe following students helped collect the dataFriedl, Colburn, Lejano, Collin, Douille, Clarice, Moh, Jasper, Kim, Caleb, Mohebi, Suhyl, Gustafson, Afton, Nowrozani, Aria, Bolanos, Bronte, Fitzsimmons, Cory, Makhlouf, Frank, Mellor, Grace, Sabado, Mariell, Rostomian, Nairi, Beruman, RJ\u003c/p\u003e\u003ch2\u003eData Availability\u003c/h2\u003e\u003cp\u003eAvailable upon request from Norbert Reich\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAlberts, B. (2009). Redefining science education. Science, \u003cem\u003e323\u003c/em\u003e(5913), 437.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAtkinson, R.D., Mayo, M.J. (2010). Refueling the U.S. Innovation Economy: Fresh Approaches to Science, Technology, Engineering and Mathematics (STEM) Education. Rochester, NY: Social Science Research Network. Retrieved from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://itif.org/publications/2010/12/07/refueling-us-innovation-economy-fresh-approaches-stem-education\u003c/span\u003e\u003cspan address=\"https://itif.org/publications/2010/12/07/refueling-us-innovation-economy-fresh-approaches-stem-education\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlank, R. K. (2013). Science Instructional Time Is Declining in Elementary Schools: What Are the Implications for Student Achievement and Closing the Gap? \u003cem\u003eScience Education\u003c/em\u003e, \u003cem\u003e97\u003c/em\u003e(6), 830\u0026ndash;847.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eDesilver, D. (2017). U.S. students\u0026rsquo; academic achievement still lags that of their peers in many other countries. Pew Research Center. Retrieved from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science\u003c/span\u003e\u003cspan address=\"https://www.pewresearch.org/fact-tank/2017/02/15/u-s-students-internationally-math-science\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFoltz, L.G., Gannon, S.,and Kirschmann, S.L., (2014) Factors that contribute to the persistence of minority students in STEM fields: this exploratory study examined factors that contributed to the college persistence of minority students in STEM graduate programs at LMCU, providing nuance and texture to the existing theory and research. \u003cem\u003ePlan High Educ\u003c/em\u003e. \u003cem\u003e42\u003c/em\u003e(4), 46\u0026ndash;59.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHossain, M.M., and Robinson, M.G. (2012). How to Motivate US Students to Pursue STEM (Science, Technology, Engineering and Mathematics) Careers. US-China education review, \u003cem\u003e2\u003c/em\u003e, 442\u0026ndash;451.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHughes, B. E. (2018). Coming out in STEM: Factors affecting retention of sexual minority STEM students. \u003cem\u003eScience Advances\u003c/em\u003e, \u003cem\u003e4\u003c/em\u003e(3), eaao6373.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKafka, A.L., Ebel, J.E., Barnett, M., Macherides-Moulis, A., Campbell, L., Gordon, S.E. (2006). Classroom Seismographs and the Challenge of Encouraging a Culture of Scientific Inquiry in K-12 Schools. \u003cem\u003eSeismology Research Letters\u003c/em\u003e, \u003cem\u003e77\u003c/em\u003e(6), 734\u0026ndash;8.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eClementson, J.J. (1991). A qualitative study of elementary teachers\u0026rsquo; and students\u0026rsquo; interactions with Full Option Science System: FOSS. (Doctoral Dissertation). University of Nebraska - Lincoln, Lincoln, Nebraska, United States.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFoster, K.M., Bergin, K.B., McKenna, A.F., Millard, D.L., Perez, L.C., Prival, J.T., Rainey, D.Y., Sevian, H.M., VanderPutten, E.A., Hamos, J.E. (2010). Partnerships for STEM Education. \u003cem\u003eScience\u003c/em\u003e, \u003cem\u003e329\u003c/em\u003e(5994), 906\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eFreeman, S., Eddy, S.L., McDonough, M., Smith, M.K., Okoroafor, N., Jordt, H., Wenderoth, M.P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences of the United States of America, \u003cem\u003e111\u003c/em\u003e(23), 8410\u0026ndash;8415.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaywood, B. K., and Besley, J. C. (2014). Education, outreach, and inclusive engagement: Towards integrated indicators of successful program outcomes in participatory science. Public Understanding of Science, \u003cem\u003e23\u003c/em\u003e(1), 92\u0026ndash;106.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIllingworth, S.,and Roop, H. (2015). Developing Key Skills as a Science Communicator: Case Studies of Two Scientist-Led Outreach Programmes. Geosciences, \u003cem\u003e5\u003c/em\u003e(1), 2\u0026ndash;14.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eIsik-Ercan, Z. (2020). \u0026lsquo;You have 25 kids playing around!\u0026rsquo;: learning to implement inquiry-based science learning in an urban second-grade classroom. \u003cem\u003eInternational Journal of Science Education\u003c/em\u003e, \u003cem\u003e42\u003c/em\u003e(3), 329\u0026ndash;49.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKoehler, B. G., Park, L. Y., \u0026amp; Kaplan, L. J. (1999). Science for Kids Outreach Programs: College Students Teaching Science to Elementary School Students and Their Parents. \u003cem\u003eJournal of Chemical Education\u003c/em\u003e, \u003cem\u003e76\u003c/em\u003e(11), 1505\u0026ndash;1509.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eLaursen, S., Liston, C., Thiry, H.,and Graf, J. (2007). What good is a scientist in the classroom? Participant outcomes and program design features for a short-duration science outreach intervention in K-12 classrooms. CBE life sciences education, \u003cem\u003e6\u003c/em\u003e(1), 49\u0026ndash;64.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRobardey, C.P., Allard, D.W., and Brown, D.M. (1994). An assessment of the effectiveness of Full Option Science System training for third-through sixth-grade teachers. \u003cem\u003eJournal of Elementary Science Education\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(1), 17\u0026ndash;29.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eTarjan, L.M., de Nesnera, K., Hoffman, R. (2015). Authentic science investigation in the classroom. Science Scope, \u003cem\u003e39\u003c/em\u003e(4), 42\u0026ndash;48.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eWilliams-Pierce, C.C. (2011). Five Key Ingredients for Improving Student Motivation. \u003cem\u003eResearch in Higher Education Journal\u003c/em\u003e, 11.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHoyer, K.M., and Sparks, D. (2017). Instructional Time for Third- and Eighth-Graders in Public and Private Schools: School Year 2011-12. Stats in Brief. NCES 2017-076. \u003cem\u003eNational Center for Education Statistics\u003c/em\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNowicki, B.L., Sullivan-Watts, B., Shim, M.K., Young, B., Pockalny, R. (2013). Factors Influencing Science Content Accuracy in Elementary Inquiry Science Lessons. \u003cem\u003eResearch in Science Education\u003c/em\u003e, \u003cem\u003e43\u003c/em\u003e(3), 1135\u0026ndash;54.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRodger W. Bybee (2014) NGSS and the Next Generation of Science Teachers, Journal of Science Teacher Education, 25:2, 211\u0026ndash;221.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNational Center for Science and Engineering Statistics. (2019). Women, Minorities, and Persons with Disabilities in Science and Engineering. Retrieved from: \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://ncses.nsf.gov/pubs/nsf19304/digest\u003c/span\u003e\u003cspan address=\"https://ncses.nsf.gov/pubs/nsf19304/digest\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNoonan, R. (2017). STEM Jobs: 2017 Update. U.S. Department of Commerce Economics and Statistics Administration Office of the Chief Economist. Retrieved from \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://www.commerce.gov/sites/default/files/migrated/reports/stem-jobs-2017-update.pdf\u003c/span\u003e\u003cspan address=\"https://www.commerce.gov/sites/default/files/migrated/reports/stem-jobs-2017-update.pdf\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHarackiewicz, J.M., Smith, J.L., Priniski, S.J. (2016). Interest Matters: The Importance of Promoting Interest in Education. \u003cem\u003ePolicy Insights from the Behavioral and Brain Sciences\u003c/em\u003e, \u003cem\u003e3\u003c/em\u003e(2), 220\u0026ndash;7.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePainter, J. (2011). Autonomy, Competence, and Intrinsic Motivation in Science Education: A Self- Determination Theory Perspective. (Doctoral dissertation). University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, United States\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePalmer, R. T., Maramba, D. C., and Dancy, T. E. (2011). A Qualitative Investigation of Factors Promoting the Retention and Persistence of Students of Color in STEM. The Journal of Negro Education, \u003cem\u003e80\u003c/em\u003e(4), 491\u0026ndash;504.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSwarat, S., Ortony, A., and Revelle, W. (2012). Activity matters: Understanding student interest in school science. \u003cem\u003eJournal of Research in Science Teaching\u003c/em\u003e, \u003cem\u003e49\u003c/em\u003e(4), 515\u0026ndash;537.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSyed, M., and Chemers, M. M. (2011). Ethnic Minorities and Women in STEM: Casting a Wide Net to Address a Persistent Social Problem: Ethnic Minorities and Women in STEM. \u003cem\u003eJournal of Social Issues\u003c/em\u003e, \u003cem\u003e67\u003c/em\u003e(3), 435\u0026ndash;441.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eYanowitz, K. L., and Vanderpool, S. S. (2004). Assessing Girls\u0026rsquo; Reactions to Science Workshops. \u003cem\u003eJournal of Science Education and Technology\u003c/em\u003e, \u003cem\u003e13\u003c/em\u003e(3), 353\u0026ndash;359. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1023/B:JOST.0000045462.08497.1c\u003c/span\u003e\u003cspan address=\"10.1023/B:JOST.0000045462.08497.1c\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eErtmer, P.A., and Simons, K.D. (2006). Jumping the PBL Implementation Hurdle: Supporting the Efforts of K\u0026ndash;12 Teachers. Interdisciplinary Journal of Problem-Based Learning. \u003cem\u003e1\u003c/em\u003e(1), 40\u0026ndash;54.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHaefner, L.A., and Zembal-Saul, C. (2004). Learning by doing? Prospective elementary teachers' developing understandings of scientific inquiry and science teaching and learning. International Journal of Science Education, \u003cem\u003e26\u003c/em\u003e(13), 1653\u0026ndash;1674.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMarkowitz, D. G. (2004). Evaluation of the Long-Term Impact of a University High School Summer Science Program on Students\u0026rsquo; Interest and Perceived Abilities in Science. Journal of Science Education and Technology, \u003cem\u003e13\u003c/em\u003e(3), 395\u0026ndash;407.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eMetz, C. J., Downes, S., and Metz, M. J. (2018). The in's and out's of science outreach: assessment of an engaging new program. Advances in physiology education, \u003cem\u003e42\u003c/em\u003e(3), 487\u0026ndash;492.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNational Research Council. (2007). Taking science to school: Learning and teaching science in grades K-8. (R.A. Duschl, H.A. Schweingruber, and A.W. Shouse, Eds.) Washington, DC: National Academies Press.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNational Research Council. (2014). Developing Assessments for the Next Generation Science Standards (J. W. Pellegrino, M. R. Wilson, J. A. Koenig, \u0026amp; A. S. Beatty, Eds.). Washington, DC: The National Academies Press.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eNold, H. (2017). Using Critical Thinking Teaching Methods to Increase Student Success: An Action Research Project. \u003cem\u003eInternational Journal of Teaching and Learning in Higher Education\u003c/em\u003e, \u003cem\u003e29\u003c/em\u003e(1), 17\u0026ndash;32.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePrince, M. (2004). Does Active Learning Work? A Review of the Research. \u003cem\u003eJournal of Engineering Education\u003c/em\u003e, \u003cem\u003e93\u003c/em\u003e(3), 223\u0026ndash;31.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eQuigley, C., Marshall, J.C., Deaton, C.C.M., Cook, M.P., Padilla, M. (2011). Challenges to Inquiry Teaching and Suggestions for How to Meet Them. \u003cem\u003eScience Educator. 20\u003c/em\u003e(1), 55\u0026ndash;61.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBuffington, C., Cerf, B., Jones, C., and Weinberg, B. A. (2016). STEM Training and Early Career Outcomes of Female and Male Graduate Students: Evidence from UMETRICS Data Linked to the 2010 Census. \u003cem\u003eAmerican Economic Review\u003c/em\u003e, \u003cem\u003e106\u003c/em\u003e(5), 333\u0026ndash;338.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eEstrada, M., Burnett, M., Campbell, A. G., Campbell, P. B., Denetclaw, W. F., Guti\u0026eacute;rrez, C. G., Hurtado, S., John, G. H., Matsui, J., McGee, R., Okpodu, C. M., Robinson, T. J., Summers, M. F., Werner-Washburne, M., and Zavala, M. (2016). Improving Underrepresented Minority Student Persistence in STEM. CBE life sciences education, \u003cem\u003e15\u003c/em\u003e(3), es5.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHussar, B., Zhang, J., Hein, S., Wang, K., Roberts, A., Cui, J., Smith, M., Bullock Mann, F., Barmer, A., and Dilig, R. (2020). The Condition of Education 2020. National Center for Education Statistics.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRyan, T. G., Whitman, A. (2013). The Inequity and Effect of Standardized Literacy Testing for First Nations Students-An Ontario (Canadian) Outlook. \u003cem\u003eJournal of Educational Thought. 46\u003c/em\u003e(2), 163\u0026ndash;181\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSteenhof, N., Woods, N.N., Van Gerven, P.W.M., Mylopoulos, M. (2019). Productive failure as an instructional approach to promote future learning. \u003cem\u003eAdvances in Health Science Education\u003c/em\u003e, \u003cem\u003e24\u003c/em\u003e(4), 739\u0026ndash;49.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eHelsel, RT, Lambert, S., Dickerson, L., Strelich, J., Woods, V. and Feldwinn, D., (2021) Design of phenomenon based science outreach program and its effects on elementary students\u0026rsquo; epistemological understanding of and attitudes towards science, School Science and Mathematics, 122 74\u0026ndash;85.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eBlanchard, M., Southerland, S. A., Osborne, J. W., Sampson, V. D., Annetta, L. A., and Granger, E. M. (2010). Is inquiry possible in light of accountability? A quantitative comparison of the relative effectiveness of guided inquiry and verification laboratory instruction. Science Education, \u003cem\u003e94\u003c/em\u003e(4): 577\u0026ndash;616.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRoehrig, G.H., and Luft, J.A. (2004). Constraints experienced by beginning secondary science teachers in implementing scientific inquiry lessons. \u003cem\u003eInternational Journal of Science Education\u003c/em\u003e, \u003cem\u003e26\u003c/em\u003e(1), 3\u0026ndash;24.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eSoysal, Y., Science Teachers\u0026rsquo; Challenging Questions for Encouraging Students to Think and Speak in Novel Ways, Science and Education (2022) December 14, \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11191-022-00411-6\u003c/span\u003e\u003cspan address=\"10.1007/s11191-022-00411-6\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGoodman, L and Berntson, G., [2000] The American Biology Teacher, Vol 7 pp 473\u0026ndash;476.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eKuchynka, SL, Eaton, A., Rivera, LM, [2022] Understanding and Addressing Gender-Based Inequities in STEM: Research Synthesis and Recommendations for U.S. K-12 Education, 16, 252\u0026ndash;288.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eJennifer DeWitt \u0026amp; Louise Archer (2015) Who Aspires to a Science Career? A comparison of survey responses from primary and secondary, International Journal of Science Education, Vol. 37, No. 13, 2170\u0026ndash;2192.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eUrsula Kessels, Anke Heyder, Martin Latsch \u0026amp; Bettina Hannover, How gender differences in academic engagement relate to students\u0026rsquo; gender identity, Educational Research Volume 56, 2014 220\u0026ndash;229.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAdam V. Maltese \u0026amp; Robert H. Tai, Eyeballs in the Fridge: Sources of early interest in Science, International Journal of Science Education, 32:5, 669\u0026ndash;685.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eE. W. Jenkins \u0026amp; R. G. Pell, Me and the Environmental Challenges: A survey of English secondary school students\u0026rsquo; attitudes towards the environment, International Journal of Science Education, 28:7, 765\u0026ndash;780.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003ePotvin, P., \u0026amp; Hasni, A. (2014a). Analysis of the decline in interest towards school science and Technology from grades 5 through 11. Journal of Science Education and Technology, 23(6), 784\u0026ndash;802.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAllison Master, Gender Stereotypes Influence Children\u0026rsquo;s STEM Motivation, Child Development Perspectives, Volume 15, Number 3, 2021, Pages 203\u0026ndash;210.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eChristine D. Clayton, Teaching for Ownership in the Middle School Science Classroom: Towards Practical Inquiry in an Age of Accountability, Teaching for Ownership MGRJ Vo. 4 (4) 53\u0026ndash;79.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRichard E. Mayer, Rote Versus Meaningful Learning (2002) THEORY INTO PRACTICE, Volume 41, Number 4, 41:4, 226\u0026ndash;232.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eRaphael, J., Tobias, S., Greenberg, R. (1999). Research Experience as a Component of Science and Mathematics Teacher Preparation. \u003cem\u003eJournal of Science Teaching in Education\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e(2), 147\u0026ndash;58.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eGraves, C., and Rutherford, S. (2012). Writing a Scientific Research \u0026ldquo;Testable\u0026rdquo; Question: the first step in using online data sets for guided inquiry assignments. Journal of College Science Teaching, \u003cem\u003e41\u003c/em\u003e, 46\u0026ndash;52.\u003c/span\u003e\u003c/li\u003e \u003cli\u003e\u003cspan\u003eAn Verburgh, S. Francois, J. Elen and R. Janssen (2013) The Assessment of Critical Thinking Critically Assessed in Higher Education: A Validation Study of the CCTT and the HCTA, Education Research International Volume 2013, Article ID 198920, 13 pages.\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":true,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"critical thinking, science outreach, middle school, science outreach","lastPublishedDoi":"10.21203/rs.3.rs-4736519/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-4736519/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWe examined the effectiveness of the SciTrek outreach program involving two junior high teachers in the same school. SciTrek has run modules in 2nd -8th grade classrooms since 2010, with each classroom participating in two modules per academic year. For this study, students (206) in eight classrooms took part in one 7th grade module focused on conservation of mass lasting six days during the Fall, and a different module focused on cellular respiration and lasting eight days in the Winter. Students took a pre-assessment prior to the modules to determine their ability to identify a testable question, to plan and design an experimental investigation, to analyze and interpret data with computational thinking. Upon finishing both modules, students took the same assessment. Students improved significantly in all aspects, showing improvements of 10\u0026ndash;45%. Students with lower academic performance (College Preparatory, three classrooms) improved more and showed more variability in their improvement than those with higher prior academic performance (Honors). The population included socioeconomically disadvantaged (57%) and Hispanic or Latino (64%) students. We discuss the importance of key features of the approach: the inclusion of four to six trained university volunteers in each classroom to guide small group discussions, and the active engagement of the 7th grade students in developing their own questions, designing their own experiments, data collection, and the defense of their conclusion to their peers.\u003c/p\u003e","manuscriptTitle":"Improving 7th grade students’ understanding of how science works","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2024-09-25 16:25:08","doi":"10.21203/rs.3.rs-4736519/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"69e8326c-f052-482d-bf72-4dbd68be65c5","owner":[],"postedDate":"September 25th, 2024","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[],"tags":[],"updatedAt":"2024-10-09T12:38:59+00:00","versionOfRecord":[],"versionCreatedAt":"2024-09-25 16:25:08","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-4736519","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-4736519","identity":"rs-4736519","version":["v1"]},"buildId":"qtupq5eGEP_6zYnWcrvyt","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}