A Reliability Generalization Meta-Analysis of the Science Motivation Questionnaire II

A Reliability Generalization Meta-Analysis of the Science Motivation Questionnaire II | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Systematic Review A Reliability Generalization Meta-Analysis of the Science Motivation Questionnaire II Dilek Karaca, Bahadır Namdar, Fulden Güler Nalbantoğlu, Begümay Tümer, and 1 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6419938/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract Science motivation is among the most important constructs affecting students’ science learning and scientific thinking skills. One commonly utilized scale to measure students’ science motivation is the Science Motivation Questionnaire II (SMQ-II), a 25-item, Likert-type, self-report scale. This study aimed to conduct a reliability generalization meta-analysis of the scale considering the REGEMA guideline. Studies included in the analysis implemented the instrument between 2011–2024 and reported a Cronbach alpha value. Reliability evidence from 49 studies reporting a coefficient for the total score or at least for one of the five subscale scores was analyzed using a random effects model and Bonnet’s transformation. The pooled Cronbach alpha reliability coefficient was 0.94 for the total score and ranged from 0.83–0.88 for the subscale scores. Moderator analyses showed generally similar reliability estimates across studies, with different study types, languages, sample types, or sample sizes for the subscales. However, the test version, sample type, sample size, and female representation in the sample showed differences in reliability estimates for the total score. The empirical evidence from the SMQ-II’s first 13 years reports high internal consistency across the scale’s scores. Educational Psychology Reliability generalization Science Motivation Science Education Meta-analysis Figures Figure 1 Figure 2 Figure 3 Figure 4 Figure 5 Figure 6 Figure 7 Figure 8 Background One of the fundamental goals of science education is to cultivate scientifically literate citizens who have a solid understanding of scientific knowledge and can use evidence-based reasoning to make informed decisions about issues related to science and society (OECD, 2007; Roberts, 2007 ). However, recent surveys indicate that students of varying ages lack basic scientific literacy (Aguilera & Perales-Palacios, 2018). Researchers have highlighted a bidirectional relationship between the enhancement of scientific literacy and science motivation (Van Vo & Csapó, 2022 ). Given the important role of motivation to learn science, several studies have explored how to examine the level of motivation of college, secondary, and elementary students (Breland et al., 2023; Huda & Rohaeti, 2023; Jian-Xin et al., 2023 ; Meulenbroeks et al., 2024 ; Shin et al., 2023 ). Thus, researchers have utilized the Science Motivation Questionnaire-II (SMQ-II) developed by Glynn et al. ( 2011 ) for different purposes. For example, the SMQ-II has been employed in conjunction with other scales to develop a model and identify factors for STEM career interest formation among students (Alexopoulos et al., 2021; Lei, 2024; Riccitelli, 2015), as well as to evaluate undergraduate students’ perceived motivation and engagement in game theme-based learning (Bónus et al., 2024; Moro et al., 2023; Watt, 2020). Other studies have employed the SMQ-II to investigate motivation to learn specific science disciplines, such as chemistry and physics (Ardura & Pérez-Bitrián, 2018; Chen et al., 2023; de Souza et al., 2022; Hibbard et al., 2016; Kahraman, 2023; Kwarikunda et al., 2021; Lee & Mun, 2023; Lei, 2024; Salta & Koulougliotis, 2015, 2020; Schönfelder & Bogner, 2020). The instrument has been translated and adapted to different languages and cultural contexts to apply to Chinese (Dong et al., 2020), Spanish (Ardura & Pérez-Bitrián, 2018), Greek (Salta & Koulougliotis, 2020), German (Schumm & Bogner, 2016), Korean (Rachmatullah & Ha, 2019), and Turkish (Tosun, 2013 ) students. Researchers have used the SMQ-II to assess students’ motivation in a fourth-year technical aircraft systems course (Ng & Chu, 2021 ), measure dental students’ oral medicine learning motivation in Indonesia, and investigate the learning motivation profile of Japanese pharmacy students (Rahmayanti et al., 2020 ). The scale was also used to determine how different motivations of learners relate to their engagement in a massive open online astronomy course on the Coursera platform (Formanek et al., 2019). Consequently, discipline-specific versions of the SMQ-II emerged as researchers adapted the instrument for their studies. For instance, the Chemistry Motivation Questionnaire II (CMQ-II) (Austin et al., 2018) and Physics Motivation Questionnaire II (PMQ-II) (Kwarikunda et al., 2020) are frequently used versions of the original questionnaire that replace the word “science” with “chemistry” or “physics,” respectively. The SMQ-II (Glynn et al., 2011 ) represents a revised version of an instrument presented in an earlier study by Glynn et al. ( 2009 ). The first version of the scale consisted of 30 items in five subscales: intrinsic motivation and personal relevance (ten items), self-efficacy and assessment anxiety (nine items), self-determination (four items), career motivation (two items), and grade motivation (five items). Two years later, the researchers modified the questionnaire to improve the construct validity, based on the results of the first version of the questionnaire, focus group interviews, expert opinions, and a pilot study. Sixteen items remained unchanged in the revised version, and nine were added. As a result, the SMQ-II utilized in this meta-analysis consists of 25 items with five subscales: intrinsic motivation (IM), self-determination (SD), self-efficacy (SE), career motivation (CM), and grade motivation (GM). Five items in each subscale are scaled on a five-point Likert scale (from never to always). In the original validation study, reliability was reported as 0.92 for the total score, 0.92 for CM, 0.81 for GM, 0.89 for IM, 0.88 for SD, and 0.83 for SE. Reliability and Reliability Generalization (RG) Reliability refers to the degree of consistency and stability of the scores from a measurement tool (Crocker & Algina, 1986 ). Errors in the measurement process reduce consistency; the source of error may vary from the scale itself, the sample (e.g., age, gender, motivation, etc.), the way the scale is applied, or the scoring complexity (Barnes et al., 2002 ; Dawis, 1987 ; Thompson & Vacha-Haase, 2000 ). It is critical to report reliability to support the reproducibility of research findings, particularly in the case of scores from a scale that is widely used in different languages and cultures, different sample groups, and different disciplines, such as the SMQ-II. The most appropriate option that allows a quantitative summary of reliability coefficients is the reliability generalization (RG) meta-analysis proposed by Vacha-Haase ( 1998 ). RG studies are meta-analyses that calculate an average reliability estimate and can explain the heterogeneity of the coefficients (Henson & Thompson, 2002 ; Rodriguez & Maeda, 2006 ; Sánchez-Meca et al., 2013 ; Vacha-Haase & Thompson, 2011 ). RG analysis also allows the examination of moderator variables that can explain a heterogeneity between the reported reliability coefficients (Sánchez-Meca et al., 2013 ). In other words, RG meta-analysis aims to estimate the average reliability of test scores based on various test applications in different samples and contexts, as well as evaluating the consistency across test applications. In addition, if differences in reliability coefficients are observed, it allows the determination of study and sample characteristics that may explain these differences from a statistical perspective (Thompson, 2002 ; Vacha-Haase, 1998 ). The SMQ-II is a scale that exhibits strong psychometric properties and has become widely used internationally, after being adapted to more than ten languages to apply to various sample groups across cultures. Since the development of the SMQ-II, studies have been conducted on its construct validity (Glynn et al., 2011 ), measurement invariance for characteristics like gender and culture (Dong et al., 2020; Toma et al, 2023), construct validity in different languages and disciplines (Ardura & Pérez-Bitrián, 2018; de Souza et al., 2022; Toma et al., 2023) and adaptation (Salta & Koulougliotis, 2015). However, to our knowledge, no researchers have conducted an RG meta-analysis of the scale. Therefore, this study aims to conduct an RG meta-analysis on the SMQ-II scale due to its widespread use in different languages and cultures, its translations and adaptations into many languages, and its frequent application in the literature. The purpose of this study was to (a) determine a precise estimate of the overall reliability coefficient for the SMQ-II; (b) investigate how reliability coefficients vary; (c) determine whether there is heterogeneity between reliability estimates, and if so, conduct moderator analyses to identify study characteristics that could explain the variability. Method Selection Criteria To improve its reporting, this paper follows the Reliability Generalization Meta Analysis (REGEMA), a 30-item checklist by Sánchez-Meca et al. ( 2021 ), which we provide as a supplement to this paper. First, a set of selection inclusion criteria was created to identify the studies for inclusion in the meta-analysis. Specifically we looked for studies that: (a) used the original SMQ-II (Glynn et al., 2011 ) or any of its adapted versions (Ardura & Pérez-Bitrián, 2018; Salta & Koulougliotis, 2020), (b) were published or unpublished (e.g., Ph.D. dissertations) between 2011–2024, (c) were written in English, (d) allowed access to full text, (e) reported Cronbach’s alpha reliability coefficient for the total SMQ-II or any of its subscales, and (f) focused on the target population in STEM fields regardless of age (e.g., university students, high school students, etc.). We also set exclusion criteria, specifically excluding studies that: (a) employed the earlier version of the SMQ (Glynn et al., 2009 ), (b) reported alpha values only from the original scale or from previous studies, (c) reported a reliability coefficient other than Cronbach’s alpha, (d) were book chapters or conference proceedings, (e) added different items to total SMQ-II or any of its subscales, (f) focused on a field other than science education. Search Strategies Articles were obtained for the study sample by querying the Web of Science, ERIC, Scopus, and ProQuest databases with the search terms Science Motivation Questionnaire II and SMQ II . The search was limited to articles published in or after 2011, the when the SMQ-II scale was first used. We ensured search consistency across the research team by first searching for a certain number of articles together, before completing separate searches in different databases. After the random assignment of electronic databases to four authors, a folder was created for each database. The abstracts of the selected publications were read and evaluated, and if they met the criteria, the full text was examined. All related documents were collected in a main file and duplicated entries were easily detected and deleted. Data Extraction and Procedure A coding form was created following the RG meta-analysis literature as suggested by Henson and Thompson ( 2002 ), and the nature of the test was taken into consideration. We extracted Cronbach’s alpha for the SMQ-II from the studies that reported at least one reliability estimate for the total score or subscale scores along with the following potential moderators: (a) study type ( article or dissertation/thesis) ; (b) language of the SMQ-II utilized ( English or other ); (c) study sample ( college students or other ); (d) test version ( original or adapted for a discipline ); (e) sample size; and (f) gender distribution (percentage of females). The language of the SMQ-II utilized was coded into two categories. The studies in the dataset adapted the SMQ-II scale into twelve different languages; however, there was not enough data for each language category. Therefore, if the SMQ-II was delivered in English, the language moderator was coded as English or otherwise coded as other . Due to a similar lack of data for each category in the data set, the test version was also coded into two categories. If the scale was presented in its original form as developed by Glyn et al. (2009) where the word science was used, the test version moderator was coded as original ; if it was adapted to another discipline (physics, chemistry, etc.), it was coded as adapted . In some studies, more than one Cronbach alpha value was reported for more than one sample (experimental-control groups). If the reported values ​​were calculated separately for different samples, they were coded separately. However, other studies reported more than one alpha value for the same sample (Ardura & Pérez-Bitrián, 2018; Salta & Koulougliotis, 2015). In these studies, although alpha values were ​​calculated separately for different groups, there was also an alpha value for the combined sample where the groups were merged. In this case, only the alpha values ​​for each group were included in the analysis. Some studies also added or removed items from the total or subdimensions of the scale (Park & ​​Kim, 2021; Zhdanov et al., 2022). The valid item numbers and reliability values ​​were coded in case of item removal. However, if new items were added to the scale, these data were not included in the analysis. Searching for SMQ-II from 2011–2024 yielded a total of 259 studies, 76 of which were duplicate publications. Of the 183 studies included in the full-text review, 78 studies were excluded from the analysis because they did not meet the eligibility criteria: they utilized SMQ rather than SMQ-II, reported prior Cronbach’s alpha values rather than computing, were not written in English, or conducted research in disciplines other than science (e.g., aviation, medicine). Thus, 49 studies that met the inclusion criteria were included in the analysis. The REGEMA flow chart in Fig. 1 presents the selection process details. Reliability induction is a type of publication bias specific to RG meta-analysis, involving reporting a reliability coefficient from prior research but not for the study itself (Vacha-Haase et al., 2000 ). In our study, 30 of the 183 eligible studies did not report the reliability value, and nine reported the reliability value from the original SMQ-II study, resulting in a reliability induction rate of 21.31%. The reliability induction rate was obtained as an average of 78.6% in a systematic study of 100 RG meta-analyses, including more than 40,000 empirical studies in psychology (Sánchez-Meca et al., 2015). Hence, it can be stated that our study’s reliability induction rate was at a reasonable level. Intercoder Agreement Three members of the research team coded the studies independently after a consensus meeting where three different studies were examined and discussed together. Then, the coders coded randomly selected ten studies that reported 60 reliability coefficients, to calculate Krippendorf’s Alpha coefficient for intercoder agreement. This analysis was performed using the irr package (Gamer et al., 2019 ) in the R statistical software (R Core Team, 2025 ). The intercoder reliability coefficient for the three coders was 0.99, indicating near-perfect agreement. Data Analysis Similar to Sandoval-Lentisco et al. ( 2023 ), we conducted separate RG meta-analyses for each SMQ-II subscale before analyzing total scores. A random effects model was utilized, as we assumed that the included reliability coefficients were a random sample from the field. All statistical analyses were conducted using the metafor package in R (Viechtbauer, 2010 ). Beretvas and Pastor ( 2003 ) suggest that alpha values reported in studies using the specified scale should not be used directly because they are usually negatively skewed. Therefore, we conducted the analyses by transforming the alpha values. We first computed outcome measures and sampling variances based on Bonnet’s transformation (2002) for the alpha values using -ln(1 - α i ), where α i is the reported reliability coefficient. We then employed the random effects model with the restricted maximum likelihood (REML) and back-transformed the results using transf = transf.iabt argument to compute 1–1/e (estimate) and reported point estimates and confidence intervals. We examined Q statistics and I 2 values ​to determine the heterogeneity between alpha coefficients. Statistically significant Q statistics (Cochran, 1954 ) and large I 2 values ​​(> 75%) are accepted as an indication of heterogeneity (Higgins et al., 2003 ). To determine the reasonable reliability estimate interval of the studies, 95% confidence intervals were calculated, and forest plots containing these confidence intervals were created to provide a visual inspection of the effect sizes of the studies. For categorical and continuous moderator variables, heterogeneity analyses were performed using univariate meta-regression models in the metafor package. This RG meta-analysis was conducted with 49 studies: 40 articles (81.63%) and 9 theses (18.37%). The studies were conducted in over 20 countries, including the USA (the majority at 36.73%), Germany, Indonesia, Brazil, China, Greece, and Turkey. Thirty-six studies applied the SMQ-II scale in English (73.47%), while 13 studies applied it in eleven different languages (26.53%), including Portuguese, Greek, German, Chinese, Spanish, Russian, and Turkish. While 44.90% of the studies involved university students, 55.10% of the sample consisted of students at lower educational levels. The studies obtained data from 21,046 people in total. Since not all the included studies reported reliability coefficients for both the total score and for each subscale score, the number of reliability coefficients in each model varied, as explained in the results section. Results Publication Bias We utilized funnel plots, Egger regression test (Egger et al., 1997), trim and fill method, and RG fail-safe N (Howell & Shields, 2008) analysis to examine publication bias for the coefficients of SMQ-II total scores. Egger’s regression test showed a statistically significant asymmetry ( t =-3.530, sd=24, p =.002), possibly due to a small study effect or possible publication bias. However, the trim and fill method analysis determined that there were no missing studies on the right side (see supplementary). The funnel plot in Figure 2 shows a symmetrical distribution, but with some asymmetry due to points remaining outside in the upper right region. Following Howell and Shields (2008), we first computed the lower bound estimate of unweighted alpha for the total score in 26 included studies. The average alpha for these 26 studies was .93 with a standard deviation of .03, with a plausible worst-case reliability coefficient of .80 standard deviations below the average. For the 134 eligible but not useful studies, average alpha was estimated as .906, hence resulting in a lower bound estimate of .91. Again, following Howell and Shields (2008) with similar criteria and a threshold reliability of .80, the RG fail-safe N was computed as 39. In other words, the mean population reliability of SMQ-II scores would be less than .80 if there were 39 file drawer studies with an unweighted average reliability of .712, a value far below the estimated lower bound. Overall, the findings obtained in the following RG meta-analysis are expected to be highly robust to publication bias. Mean Reliability and Heterogeneity Table 1 reports the year, type, and country for the included studies. Forty of the studies were articles, whereas nine were theses. Although the studies conducted in more than 20 countries, most were conducted in the USA (36.73%). For the subscales, 47 alpha values were obtained from 32 different studies for CM, 45 from 33 studies for GM, 43 from 32 studies for IM, 46 from 31 studies for SD, and 50 from 34 studies for SE. For the SMQ-II total score, 27 values were obtained from 23 different studies. The data analyzed had been collected from 12,434 individuals for the CM subscale, 12,683 for GM, 16,211 for IM, 11,559 for SD, 13,675 for SE, and 6,881 individuals from more than three countries for the total scale. Figures 3–8 present forest plots of reliability coefficients for the overall scale and each subscale. The total scale reliability values of the studies ranged from .85 (Moro & Billote, 2023) to .97 (Huda & Rohaeti, 2023; Lee & Mun, 2023; Razali et al., 2020). The forest plot in Figure 8 shows that the weights of all studies are close to each other, apart from a few studies (Lei, 2024; Migalang & Azuelo, 2020; Watt, 2020). Table 2 shows the main results for the total and each of the five subscales of the SMQ-II scale. The weighted average calculated for alpha for the total score of the SMQ-II scale was 0.94 (95% CI[.93, .95]) and was statistically significant at the p <.001 level. The Q statistic was significant ( Q =664.0269, p <.001) and the I 2 statistic was 96.36. Among the subscales of the SMQ-II, α + =0.88 (95% CI:[0.86, 0.90]) for CM, α + =0.83 (95% CI:[0.80, 0.85]) for GM, α + =0.85 (95% CI:[0.82, 0. 87]), α + =0.84 (95% CI:[0.82, 0.86]) for SD, and α + =0.85 (95% CI:[0.84, 0.87]) for SE were statistically significant at p <.001 level. Cochran’s Q statistic and I 2 statistics were found to be statistically significant in the CM ( Q =1436.1580, p <.0001, I 2 =96.96), GM ( Q =1960.1004, p <.0001, I 2 =96.98); IM ( Q =1412. 9436, p <.0001, I 2 =97.40), SD ( Q =1059.1763, p <.0001, I 2 =95.88), and SE subscales ( Q =1366.1194, p <.0001, I 2 =95.95). Accordingly, the statistically significant Q statistics and high I 2 values indicate a high level of heterogeneity among the studies reporting reliability values for the subscales of the SMQ-II scale. Moderator Analysis The Q statistics and high I 2 values (>92) show significant heterogeneity between the mean alpha coefficients for the SMQ-II total score and its subscales, as supported by the forest plots. Moderator analyses were conducted to investigate potential sources of heterogeneity, with study type, study language, sample type, and test version as categorical variables, and sample size and female percentage as continuous variables. While there were missing values for continuous moderator variables in the data set, no imputation was made for these values, and they were considered missing data. The results of the meta-regression analysis for categorical moderator variables are presented in Table 3. The results show that there are no significant differences in the reliability coefficients of moderator variables in the CM, GM, and IM subscales ( p >.05). However, the test version was a statistically significant moderator variable in the SD (Q M =7.712, p =.006) and SE (Q M =9.726, p =.002) subscales. The results of the analysis revealed that the average reliability coefficients of the original test were higher than those of the adapted test. In addition, the sample type was a statistically significant moderator variable for the total scale (Q M =10.540, p =.001). The average reliability value calculated for the samples consisting of university students (0.92) was lower than the samples consisting of other students (0.95). Table 3 presents the average reliability coefficients of the moderator variables for the total scale and subscales. Table 4 presents the results of the meta-regression analysis of continuous moderator variables. The sample size variable had a statistically significant effect on the total scale ( b j =.001, SE=-.000, p =.028, R 2 =13.87). Although this finding indicates that increasing the sample size significantly affects the reliability coefficient calculated for the total scale, it also indicates that the effect is relatively small. However, the sample size does not have a statistically significant effect on the reliability coefficient of any of the subscales (p>.05). Gender distribution, one of the moderator variables, shows a statistically significant negative effect for GM ( b j =-.011, SE=.005, p =.028, R 2 = 12.08). This finding indicates that an increase in the proportion of female participants has a significant, but relatively small effect on the reliability coefficient for the GM subscale. For the total scale and other subscales (CM, IM, SD, and SE), gender distribution had no significant effect ( p >.05). Conclusions and Discussion This reliability generalization meta-analysis study aimed to estimate the mean reliability coefficient of the SMQ-II scale and its subscales and examine moderator variables that may cause variability between studies. For this purpose, a meta-analysis was conducted on 49 studies obtained within the scope of the criteria determined. The mean reliability coefficient for the SMQ-II total scale score was calculated as 0.94. This value is very close to the original reliability coefficient of 0.92 reported by Glynn et al. (2011), but still slightly higher. The mean reliability coefficients of the SMQ-II subscales were between 0.83–0.88, which can be expressed as “very good” according to DeVellis (2003). When compared with the original reliability coefficients of the scale, the mean reliability coefficient was higher in two subscales (GM and SE) and lower in three subscales (CM, IM, and SD). A large heterogeneity ( I 2 =96.36) was found between the coefficients for the SMQ-II total scale. Thus, we recommend that researchers who plan to use the scale calculate the reliability value for their own samples and not make reliability induction. Our results also showed that the reliability values for the total scale varied according to the type of sample (university versus lower-level students). The mean reliability value calculated for samples consisting of university students (0.92) was lower than that for samples consisting of lower-level students (0.95). While the scientific motivation of university students is related to individual interest and academic/career goals, it may depend on teacher or parental guidance for students at other educational levels. Second, students’ levels of intellectual and cognitive development may affected their responses to the scale. Less advanced students may have perceived and made sense of abstract concepts such as scientific motivation differently than more advanced students. The result that the sample type affects the average reliability estimate mirrors the findings of previous studies by Vacha-Haase (1998), Yin and Fan (2000), Eser and Dogan (2023), and Batı and Irmak (2024). Another important moderator variable for total score reliability was sample size. An increase in sample size significantly affected the reliability coefficient, but this effect was statistically expected, as large samples provide more stable results than small samples. The positive relationship between sample size and reliability coefficients has yielded similar results in other studies (Sandoval-Lentisco et al., 2023). Significant heterogeneity was also found between the reliability coefficients in the subscales. The moderator analyses to gauge heterogeneity concluded that the type of study, language, sample size, and sample type variables did not affect the subscale reliability values; however, test version, one of the moderator variables, did impact the change of reliability values in the SD and SE subscales. Moreover, the average reliability coefficients of the original test version were higher than the adapted test version. Another moderator variable, female percentage, was found to have an effect on the reliability values in the GM subscale. However, this effect was relatively small. This result could stem from the differing levels of gender equality in the countries where the studies in the sample were conducted. Earlier measurement invariance studies for the SMQ-II did not reveal significant differences between genders (Dong et al., 2020; Toma et al., 2023). However, our results suggest that new studies should employ measurement invariance studies to assure that the SMQ-II measures similarly across genders. The present study’s analysis of research conducted on individuals from more than 15 countries, with sample sizes ranging from 6,800 to 16,000, revealed very high reliability coefficients for both the total and subscale scores of the SMQ-II. However, reliability is not a fixed characteristic of scales, rather a characteristic of the scores obtained at the end of the measurements (Crocker & Algina, 1986). Organizations such as the APA (Wilkinson, 1999), American Educational Research Association, and National Council on Measurement in Education recommend that researchers calculate and report the reliability values of their own studies. The literature suggests that many researchers do heed this recommendation and present the scale either by referring to the original reliability coefficient or the reliability coefficient obtained in another study, which Vacha-Haase et al. (2000) refer to as “reliability induction.” Therefore, it is important for researchers to report the reliability coefficients obtained from their own studies, regardless of which scale they use. Investigations have shown that researchers’ most commonly reported reliability coefficient is Cronbach’s alpha (Scherer & Teo, 2020). There are several potential reasons for this. First, Cronbach’s alpha is easier to calculate than other reliability coefficients and can be calculated directly through many statistical programs. Second, researchers may prefer it because it can be calculated without multiple test administrations (Scherer & Teo, 2020). Third, calculating other reliability coefficients may require more complex statistical knowledge and computational skills. It may also be that these methods are not well known by researchers, or that academic journal reviewers and editors are more familiar with Cronbach’s alpha and demand this coefficient. Therefore, researchers may be inclined to report Cronbach’s alpha reliability coefficient to minimize potential difficulties in the publication process. Implications and Limitations López-Ibáñez et al. (2024) analyzed the RG meta-analysis studies conducted between 1998–2020 and reported that Cronbach’s alpha was dominant among the reliability coefficients used, despite criticisms (Yang & Green, 2011). In this context, it is recommended that researchers turn to alternative reliability values and report these values along with Cronbach’s alpha to evaluate reliability. In our RG meta-analysis of the SMQ-II, the type of study, study language, sample type, test version, sample size, and female percentage were identified as moderator variables to determine the sources of heterogeneity. Future studies can focus on different moderator variables to investigate the sources of heterogeneity. More in-depth analyses are needed, especially for sample types that show statistically significant differences across sources of heterogeneity. Future research could focus on systematically examining how reliability coefficients vary across different age groups, educational levels, and cultural contexts. Such studies would make important contributions to assessing the applicability of the scales across different demographic and socio-cultural groups. The SMQ-II has been translated into many languages and modified for use in a variety of fields, including biology, chemistry, physics, and aviation. However, our study only considered English-language research, which poses another limitation since it results in the exclusion of research in other languages. At the same time, although there are adaptation studies in many disciplines, the test version variable of the scale could not be examined separately according to these disciplines due to the limited number of studies in the data set. In our study, the studies using the original form of the scale were coded as “original” in the test version, and the versions adapted to different disciplines were coded as “adapted” and included in the analysis. In this context, the inability to conduct detailed analyses on a discipline-by-discipline basis regarding the translation and adaptation processes of the scale also creates a limitation. Since we only reviewed studies that applied the SMQ-II to science education, future RG meta-analyses of the scale should include studies in different disciplines (e.g., aviation or medicine) and languages. Detailed reporting of different reliability coefficients, sample characteristics, and descriptive statistics of the sampled studies would allow future research to focus on new coefficients and moderator variables. Technological advances and digital learning environments affect student motivation towards science and may consequently change how students respond to tests and questionnaires. In this context, differences can be observed between the reliability coefficients obtained from SMQ-II assessments applied in traditional paper-and-pencil form and the coefficients obtained from digital assessments. However, there is not enough data in the studies examined in the present research to evaluate this difference. We recommend that this situation be systematically addressed in future RG meta-analysis studies. Despite the aforementioned limitations, it is still reasonable to assert that the SMQ-II scale has strong internal consistency and robustness to adaptations for related disciplines. References Aguilera, D., & Perales-Palacios, F. (2020). What effects do didactic interventions have on students’ attitudes towards science? A Meta-Analysis. Research in Science Education, 50, 573–597. https://doi.org/10.1007/s11165-018-9702-2 *Alexopoulos, A. N., Paolucci, P., Sotiriou, S. A., Bogner, F. X., Dorigo, T., Fedi, M., ... & Scianitti, F. (2021). The colours of the Higgs boson: a study in creativity and science motivation among high-school students in Italy. Smart Learning Environments , 8 , 1–23. https://doi.org/10.1186/s40561-021-00169-4 *Ardura, D., & Pérez-Bitrián, A. (2018). The effect of motivation on the choice of chemistry in secondary schools: adaptation and validation of the Science Motivation Questionnaire II to Spanish students. Chemistry Education Research and Practice , 19 (3), 905–918. https://doi.org/10.1039/C8RP00098K *Austin, A. C., Hammond, N. B., Barrows, N., Gould, D. L., & Gould, I. R. (2018). Relating motivation and student outcomes in general organic chemistry. Chemistry Education Research and Practice , 19 (1), 331–341. https://doi.org/10.1039/C7RP00182G Barnes, L. L., Harp, D., & Jung, W. S. (2002). Reliability generalization of scores on the Spielberger state-trait anxiety inventory. Educational and psychological measurement , 62 (4), 603–618. https://doi.org/10.1177/0013164402062004005 Batı, K., & Irmak, Ş. (2024). A reliability generalization study of the STEM-CIS scale: Exploring moderator effects. Current Psychology, 43 , 35453–35470. https://doi.org/10.1007/s12144-024-07036-8 Beretvas, S. N., & Pastor, D. A. (2003). Using mixed-effects models in reliability generalization studies. Educational and Psychological Measurement , 63 (1),75–95.https://doi.org/10.1177/0013164402239318 *Bernard, R. A. (2018). The relationship of formal reasoning, motivation, and conceptual change: a quantitative study of introductory biology students across the United States (Doctoral dissertation, Montana State University). Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics , 27 (4), 335–340. https://doi.org/10.3102/10769986027004335 *Bónus, L., Antal, E., & Korom, E. (2024). Digital game-based inquiry learning to improve eighth graders’ inquiry skills in biology. Journal of Science Education and Technology , 1–17. https://doi.org/10.1007/s10956-024-10096-x *Breland, H., Clark, C. M., Shaked, S., & Paquette-Smith, M. (2023). The benefits of participating in a learning assistant program on the metacognitive awareness and motivation of learning assistants. CBE—Life Sciences Education , 22 (3), ar30. https://doi.org/10.1187/cbe.22-08-0156 *Chen, Y. C., Terada, T., & Aguirre-Mendez, C. (2023). Action research to explore argumentative writing projects supported with online simulation for developing conceptual knowledge and motivation to learn. Reading and Writing , 36 (9), 2275–2317. https://doi.org/10.1007/s11145-022-10363-6 *Christiansen, M. D. (2015). Science as a direct measure of academic hope: the validation of the science hope scale . (Doctoral dissertation, Hofstra University). *Chumbley, S., Hainline, M. S., Wells, T., & Haynes, J. C. (2022). Students' motivation to learn science through undergraduate-level agricultural coursework. Journal of Agricultural Education , 63 (1), 182–199. https://doi.org/10.5032/jae.2022.01182 Cochran, W. G. (1954). The combination of estimates from different experiments. Biometrics , 10 (1), 101–129. *Conner, L. C. (2021). Impact of flipped learning on high school students’ motivation, cognitive load, and perceptions of flipped approach for learning science (Doctoral dissertation, University of South Carolina). Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory . Holt, Rinehart and Winston. Orlando, FL. Dawis, R. V. (1987). Scale construction. Journal of Counseling Psychology, 34, 481–489. *de Souza, R. T., da Silva, M. D. S. B., Barbato, D. M., de Guzzi, M. E. R., & Kasseboehmer, A. C. (2022). Motivation to learn chemistry: a thorough analysis of the CMQ-II within the Brazilian context. Chemistry Education Research and Practice , 23 (4), 799–810. https://doi.org/10.1039/D2RP00107A DeVellis, R. F. (2003). Scale Development: Theory and Application. Thousand Oaks, CA: Sage. *Dixon, K. L. (2017). The effect of the flipped classroom on urban high school students' motivation and academic achievement in a high school science course (Doctoral dissertation, Liberty University). *Dixon, K., & Wendt, J. L. (2021). Science motivation and achievement among minority urban high school students: An examination of the flipped classroom model. Journal of Science Education and Technology , 30 (5), 642–657. https://doi.org/10.1007/s10956-021-09909-0 *Dong, Z., Li, M., Minstrell, J., & Cui, Y. (2020). Psychometric properties of Science Motivation Questionnaire II‐Chinese version in two waves of longitudinal data. Psychology in the Schools , 57 (8), 1240–1256. https://doi.org/10.1002/pits.22370 Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. bmj , 315 (7109), 629–634. Eser, M. T., & Doğan, N. (2023). Life Satisfaction Scale: A meta-analytic reliability generalization study in Turkey sample. Turkish Psychological Counseling and Guidance Journal , 13 (69), 224–239. https://doi.org/10.17066/tpdrd.1223320mn *Formanek, M., Buxner, S., Impey, C., & Wenger, M. (2019). Relationship between learners’ motivation and course engagement in an astronomy massive open online course. Physical Review Physics Education Research , 15 (2), 020140. https://doi.org/10.1103/PhysRevPhysEducRes.15.020140 Gamer, M., Lemon, J., Fellows, I., & Singh, P. (2019). irr: Various coefficients of interrater reliability and agreement (Version 0.84.1) [R package]. Comprehensive R Archive Network. https://CRAN.R-project.org/package=irr Glynn, S. M., Taasoobshirazi, G., & Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience majors. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching , 46 (2), 127–146. https://doi.org/10.1002/tea.20267 Glynn, S. M., Brickman, P., Armstrong, N., & Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. Journal of research in science teaching , 48 (10), 1159–1176. https://doi.org/10.1002/tea.20442 *Gomes de Freitas, R. T., Tolentino Cicuto, C. A., & Pazinato, M. S. (2020). Motivation to learn natural sciences in the context of a training of teacher’s course. Problems of Education in the 21st Century , 78 (3), 359–370. http://dx.doi.org/10.33225/pec/20.78.359 Jian-Xin, Y., Yu-Xuan, X., Tian, L., Chu-Fan, D., Yu-Ying, G., & Fortus, D. (2023). Disciplinary learning motivation and its external influencing factors: Taking physics in a “selection crisis” as an example. Research in Science Education , 53 (4), 823–839. https://doi.org/10.1007/s11165-023-10112-x Hakstian, A. R., & Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika , 41 (2), 219–231. *Haviz, M., Maris, I. M., Azis, D., & Helmita, R. (2024). An investigation into science motivation, technology acceptance, and satisfaction intention during the transition to e-learning of prospective biology teachers. Cogent Education , 11 (1), 2393065. https://doi.org/10.1080/2331186X.2024.2393065 *Heim, A. B. (2020). Understanding learner-centeredness and student engagement in undergraduate biology education (Doctoral dissertation, University of Northern Colorado). Henson, R. K., & Thompson, B. (2002). Characterizing measurement error in scores across studies: Some recommendations for conducting “reliability generalization” studies. Measurement and Evaluation in Counseling and Development , 35 (2), 113–127. https://doi.org/10.1080/07481756.2002.12069054 *Hibbard, L., Sung, S., & Wells, B. (2016). Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence. Journal of Chemical Education , 93 (1), 24–30. https://doi.org/10.1021/acs.jchemed.5b00592 Higgins, J. P., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003). Measuring inconsistency in meta-analyses. Bmj , 327 (7414), 557–560. Howell, R. T., & Shields, A. L. (2008). The file drawer problem in reliability generalization: A strategy to compute a fail-safe N with reliability coefficients. Educational and Psychological Measurement , 68 (1), 120–128. *Huda, N., & Rohaeti, E. (2023). The factors that influence the motivation to learn chemistry of upper-secondary school students in Indonesia. Journal of Baltic Science Education , 22 (4), 615–630. http://dx.doi.org/10.33225/jbse/23.22.615 *Hugerat, M., Kortam, N., Maroun, N. T., & Basheer, A. (2020). The educational effectiveness of didactical games in project-based science learning among 5th grade students. Eurasia Journal of Mathematics, Science and Technology Education , 16 (10), em1888. https://doi.org/10.29333/ejmste/8490 *Iyer, R. D. (2017). Uncovering students' motivation for learning science: Gender differences in Mumbai. Global Business and Management Research , 9 (3), 16. *Kahraman, S. (2023). The use of dynamic computer visualizations integrated with the poe sequence: its effect on learners’ understanding, retention, and motivation. Canadian Journal of Science, Mathematics and Technology Education , 23 (2), 179–209. https://doi.org/10.1007/s42330-023-00284-z *Kulesza, A. E., Imtiaz, S., & Bernot, K. M. (2022). Building connections to biology and community through service-learning and research experiences. Journal of Microbiology & Biology Education , 23 (3), e00082–22. https://doi.org/10.1128/jmbe.00082-22 *Kwarikunda, D., Schiefele, U., Ssenyonga, J., & Muwonge, C. M. (2020). The relationship between motivation for, and interest in, learning physics among lower secondary school students in Uganda. African Journal of Research in Mathematics, Science and Technology Education , 24 (3), 435–446. https://doi.org/10.1080/18117295.2020.1841961 *Kwarikunda, D., Schiefele, U., Ssenyonga, J., & Muwonge, C. M. (2021). Secondary school students’ motivation profiles for physics learning: Relations with cognitive learning strategies, gender, attitudes and individual interest. African Journal of Research in Mathematics, Science and Technology Education , 25 (2), 197–210. https://doi.org/10.1080/18117295.2021.1956720 *Lawson, D. (2020). Motivating critical thinkers in fourth grade science: action research promoting 21st century skills through technology integrated project-based learning (Doctoral dissertation, University of South Carolina). *Lee, G. G., & Mun, S. (2023). From science motivation to science identity: The mediating effect of science achievement according to gender. Eurasia Journal of Mathematics, Science and Technology Education , 19 (10), em2341. https://doi.org/10.29333/ejmste/13633 *Lei, Y. (2024). Unlocking the secrets of STEM success: Exploring the interplay of motivation to learn science, self-regulation, and emotional intelligence from a perspective of self-determination theory. Learning and Motivation , 87 , 102012. https://doi.org/10.1016/j.lmot.2024.102012 López-Ibáñez, C., López-Nicolás, R., Blázquez-Rincón, D. M., & Sánchez-Meca, J. (2024). Reliability generalization meta-analysis: Comparing different statistical methods. Current Psychology , 43 (20), 18275–18293. https://doi.org/10.1007/s12144-023-05604-y Meulenbroeks, R., van Rijn, R., & Reijerkerk, M. (2024). Fostering secondary school science students’ intrinsic motivation by inquiry-based learning. Research in Science Education , 54 (3), 339–358. https://doi.org/10.1007/s11165-023-10139-0 *Migalang, G. M. M., & Azuelo, A. G. (2020). Students’ cognitive learning and motivation through hybrid instructional strategy. Journal of Science & Mathematics Education in Southeast Asia , 43 . *Moro, K. C., & Billote, W. J. S. M. (2023). Integrating Ivatan indigenous games to learning module in physics: its effect to student understanding, motivation, attitude, and scientific sublime. Science Education International , 34 (1), 3–14. https://doi.org/10.33828/sei.v34.i1.1 Ng, D. T. K., & Chu, S. K. W. (2021). Motivating students to learn STEM via engaging flight simulation activities. Journal of Science Education and Technology , 30 (5), 608–629. https://doi.org/10.1007/s10956-021-09907-2 Organization for Economic Cooperation and Development (OECD). (2007). PISA 2006: Science Competencies for Tomorrow’s World (Volume I: Analysis). Paris. Park, S., & Kim, S. (2021). Is sustainable online learning possible with gamification? The effect of gamified online learning on student learning. Sustainability , 13 (8), 4267. https://doi.org/10.3390/su13084267 *Pope, A. S. (2018). Science attrition among high-performing college students: An analysis of motivational factors to improve retention strategies (Doctoral dissertation, Tennessee State University). *Rachmatullah, A., & Ha, M. (2019). Indonesian and Korean high school students’ disparities in science learning orientations: an approach to multi-group structural equation modeling. Asia-Pacific Science Education , 5 (1), 1–17. https://doi.org/10.1186/s41029-019-0048-5 *Rachmatullah, A., Roshayanti, F., Shin, S., Lee, J. K., & Ha, M. (2018). The secondary-student science learning motivation in Korea and Indonesia. EURASIA Journal of Mathematics, Science and Technology Education , 14 (7), 3123–3141. https://doi.org/10.29333/ejmste/91665 Rahmayanti, F., Irmagita, A., Wardhany, I. I., & Gunawan, B. (2020). Measurement of oral medicine learning motivation in dental students using the Indonesian version of Science Motivation Questionnaire II (SMQ-II). Pesquisa Brasileira em Odontopediatria e Clínica Integrada , 20 , e5486. https://doi.org/10.1590/pboci.2020.136 *Razali, F. (2021). Exploring crucial factors of an interest in STEM career model among secondary school students. International Journal of Instruction, 14 (2), 385–404. https://doi.org/10.29333/iji.2021.14222a *Razali, F., Manaf, U. K. A., & Ayub, A. F. M. (2020). STEM education in Malaysia towards developing a human capital through motivating science subject. International Journal of Learning, Teaching and Educational Research , 19 (5), 411–422. https://doi.org/10.26803/ijlter.19.5.25 R Core Team (2025). A language and environment for statistical computing . R foundation for statistical computing, Vienna, Austria. https://www.R-project.org *Reece, A. J., & Butler, M. B. (2017). Virtually the same: A comparison of STEM students’ content knowledge, course performance, and motivation to learn in virtual and face-to-face introductory biology laboratories. Journal of College Science Teaching , 46 (3), 83. https://doi.org/10.2505/4/jcst17_046_03_83 *Riccitelli, M. (2015). Science identity's influence on Community College Students' engagement, persistence, and performance in biology (Doctoral dissertation, Northcentral University). Rodriguez, M. C., & Maeda, Y. (2006). Meta-analysis of coefficient alpha. Psychological Methods, 11, 306–322 . Roberts, D. (2007). Scientific literacy/science literacy. In S. K. Abell & N. G. Lederman (Eds.), Handbook of Research on Science Education (pp. 729–780). Mahwah, NJ: Lawrence. *Salta, K., & Koulougliotis, D. (2015). Assessing motivation to learn chemistry: adaptation and validation of Science Motivation Questionnaire II with Greek secondary school students. Chemistry Education Research and Practice , 16 (2), 237–250. https://doi.org/10.1039/C4RP00196F *Salta, K., & Koulougliotis, D. (2020). Domain specificity of motivation: chemistry and physics learning among undergraduate students of three academic majors. International Journal of Science Education , 42 (2), 253–270. https://doi.org/10.1080/09500693.2019.1708511 Sánchez-Meca, J., López-López, J. A. & López-Pina, J. A. (2013). Some recommended statistical analytic practices when reliability generalization studies are conducted. British Journal of Mathematical and Statistical Psychology, 66 (3), 402–425. 10.1111/j.2044-8317.2012.02057. https://doi.org/10.1111/j.2044-8317.2012.02057.x Sanchez-Meca, J., Rubio-Aparicio, M., López-Pina, J., Nuñez-Nuñez, R.M., & Marın-Martınez, F. (2015). The phenomenon of reliability induction in the social and health sciences. XIV Congress of Methodology of the Social and Health Sciences. Sánchez‐Meca, J., Marín‐Martínez, F., López‐López, J. A., Núñez‐Núñez, R. M., Rubio‐Aparicio, M., López‐García, J. J., ... & López‐Nicolás, R. (2021). Improving the reporting quality of reliability generalization meta‐analyses: The REGEMA checklist. Research Synthesis Methods , 12 (4), 516–536. https://doi.org/10.1002/jrsm.1487 Sandoval‐Lentisco, A., López‐Nicolás, R., López‐López, J. A., & Sánchez‐Meca, J. (2023). Florida Obsessive‐Compulsive Inventory and Children's Florida Obsessive Compulsive Inventory: A reliability generalization meta‐analysis. Journal of Clinical Psychology , 79 (1), 28–42. https://doi.org/10.1002/jclp.23416 Scherer, R., & Teo, T. (2020). A tutorial on the meta-analytic structural equation modeling of reliability coefficients. Psychological methods , 25 (6), 747. *Schumm, M. F., & Bogner, F. X. (2016). Measuring adolescent science motivation. International Journal of Science Education , 38 (3), 434–449. https://doi.org/10.1080/09500693.2016.1147659 *Schmid, S., & Bogner, F. X. (2017). How an inquiry-based classroom lesson intervenes in science efficacy, career-orientation and self-determination. International Journal of Science Education , 39 (17), 2342–2360. https://doi.org/10.1080/09500693.2017.1380332 *Schönfelder, M. L., & Bogner, F. X. (2020). Between science education and environmental education: How science motivation relates to environmental values. Sustainability , 12 (5), 1968. https://doi.org/10.3390/su12051968 *Shin, S., Rachmatullah, A., Ha, M., & Lee, J. K. (2018). A longitudinal trajectory of science learning motivation in Korean high school students. Journal of Baltic Science Education , 17 (4), 674–687. https://doi.org/10.33225/jbse/18.17.674 Shin, D. D., Lee, M., Jung, S. J., & Bong, M. (2023). Relative effects of classroom utility value intervention on the science motivation of girls and boys. Research in Science Education , 53 (3), 593–612. https://doi.org/10.1007/s11165-022-10070-w Thompson, B. (2002). Score reliability: Contemporary thinking on reliability issues . Sage. Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60 (2), 174–195. https://doi.org/10.1177/0013164400602 *Toma, R. B., Cordeiro Bizerra, A. M., Yánez, I., & Meneses Villagrá, J. Á. (2023). Cross-cultural adaptation of the Science Motivation Questionnaire II (SMQ-II) for Portuguese-speaking Brazilian secondary school students. Revista Latinoamericana de Psicología , 55 , 109–119. https://doi.org/10.14349/rlp.2023.v55.13 Tosun, C. (2013). Kimya Motivasyon Anketi-II’nin Türkçeye uyarlanması. Erzincan Üniversitesi Eğitim Fakültesi Dergisi , 15 (1), 173–202. Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational and Psychological Measurement , 58 (1), 6–20. Vacha-Haase, T., Kogan, L. R., & Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. Educational and Psychological Measurement , 60 (4), 509–522. https://doi.org/10.1177/00131640021970682 Vacha-Haase, T., & Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. Measurement and Evaluation in Counseling and Development, 44 , 159–168. http://doi.org/10.1177/0748175611409845 Van Vo, D., & Csapó, B. (2022). Exploring students’ science motivation across grade levels and the role of inductive reasoning in science motivation. European Journal of Psychology of Education , 37 (3), 807–829. http://dx.doi.org/10.1007/s10212-021-00568-8 Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software , 36, 1–48. *Watt, K. (2020). Guided simulation: A new model of game-based pedagogy for non-stem students in the community college (Doctoral dissertation, Gwynedd Mercy University). Wilkinson, L. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist , 54 (8), 594. *Willoughby, B. R., Flack, N. A., Bird, R. J., & Woodley, S. J. (2024). Motivation to learn in university science students studying anatomy: A mixed‐methods analysis of what drives learning. Anatomical sciences education , 17 (4), 818–830. https://doi.org/10.1002/ase.2416 Yang, Y., & Green, S. B. (2011). Coefficient alpha: A reliability coefficient for the 21st century? Journal of Psychoeducational Assessment, 29 (4), 377–392. https://doi.org/10.1177/0734282911406668 Yin, P., & Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. Educational and psychological measurement , 60 (2), 201–223. *You, H. S., Kim, K., Black, K., & Min, K. W. (2018). Assessing science motivation for college students: Validation of the Science Motivation Questionnaire II using the rasch-andrich rating scale model. Eurasia Journal of Mathematics, Science and Technology Education , 14 (4), 1161–1173. https://doi.org/10.29333/ejmste/81821 *Zhdanov, S. P., Ishmuradova, A. M., Zakharova, V. L., Belous, S. V., & Grishnova, E. E. (2022). Validation and adaptation of the questionnaire on science motivation in the Russian context. Eurasia Journal of Mathematics, Science and Technology Education , 18 (8). https://doi.org/10.29333/ejmste/12219 Note: References with an asterisk (*) indicate studies included in the reliability generalization meta-analysis in this study. Tables Tables 1 to 4 are available in the Supplementary Files section Additional Declarations The authors declare no competing interests. Supplementary Files RGMeta.r Rcode_RG_SMQII SupplementaryDocumentsRGSMQII.docx Supplementary_RG_SMQ_II Tables.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6419938","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Systematic Review","associatedPublications":[],"authors":[{"id":441168776,"identity":"58fac81f-e8c0-493f-8ae0-6454b9974763","order_by":0,"name":"Dilek Karaca","email":"","orcid":"https://orcid.org/0000-0001-8004-4211","institution":"Manisa Celal Bayar Univesity","correspondingAuthor":false,"prefix":"","firstName":"Dilek","middleName":"","lastName":"Karaca","suffix":""},{"id":441168777,"identity":"ab2261e9-92b9-4027-ab62-3fbef9c775cf","order_by":1,"name":"Bahadır Namdar","email":"","orcid":"https://orcid.org/0000-0002-5076-6034","institution":"Ege University","correspondingAuthor":false,"prefix":"","firstName":"Bahadır","middleName":"","lastName":"Namdar","suffix":""},{"id":441168778,"identity":"6e9aa815-aaef-4115-bd21-ef8a0def7315","order_by":2,"name":"Fulden Güler Nalbantoğlu","email":"","orcid":"https://orcid.org/0000-0002-1233-6666","institution":"Ege University","correspondingAuthor":false,"prefix":"","firstName":"Fulden","middleName":"Güler","lastName":"Nalbantoğlu","suffix":""},{"id":441168779,"identity":"989d4995-c078-40cb-96f9-5266b73a2716","order_by":3,"name":"Begümay Tümer","email":"","orcid":"https://orcid.org/0000-0003-4708-2034","institution":"Ege University","correspondingAuthor":false,"prefix":"","firstName":"Begümay","middleName":"","lastName":"Tümer","suffix":""},{"id":441168780,"identity":"6f981314-b547-466d-8351-47cb82f2bf10","order_by":4,"name":"Burak Aydın","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAABAklEQVRIiWNgGAWjYBACAyBmbACxDjMwPngAFWFgYCNOC7NBAmlaDjCwSRClxVwi9+HHGQx2iX3HeY9VJFQczjNnP3vwA0OZDU4tljPSjSU3MCQnzjzMl3Yj4czhYsuevGQJhnNpuB12I41B8gEDc+KGwzxmNxLbDiduOJBjIMHYdhifFuafDxjqwVoKEv8BtZx/Y/yDse0/Pi1sQIcdBmthSGwAMm7kmAFtOYDbLz3P2CxnGBw3nnmYx1gi4Vh6scGNN2YWCeeScWoxZ09jvtlTUS3bd/6M4YcPNdZ5BudzjG98KLPDqQXqPAbHBigzAYnED+xhDGIUj4JRMApGwQgDAGy6XCh0id2FAAAAAElFTkSuQmCC","orcid":"https://orcid.org/0000-0003-4462-1784","institution":"Leuphana University","correspondingAuthor":true,"prefix":"","firstName":"Burak","middleName":"","lastName":"Aydın","suffix":""}],"badges":[],"createdAt":"2025-04-10 11:59:24","currentVersionCode":1,"declarations":{"humanSubjects":false,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":false,"humanSubjectConsent":false,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-6419938/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6419938/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":80377188,"identity":"f91bb765-a5c8-4408-802d-8c7f709f23aa","added_by":"auto","created_at":"2025-04-11 08:09:01","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":116294,"visible":true,"origin":"","legend":"\u003cp\u003eREGEMA Flow Chart for SMQ-II RG Meta-analysis\u003c/p\u003e","description":"","filename":"1.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/8bc21ded8ec0d5e1d0483b2a.png"},{"id":80377177,"identity":"134ce826-aa35-42be-986c-d74438359497","added_by":"auto","created_at":"2025-04-11 08:09:01","extension":"png","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":51570,"visible":true,"origin":"","legend":"\u003cp\u003eFunnel plot of Cronbach alpha coefficients for SMQ-II total score\u003c/p\u003e","description":"","filename":"2.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/8f69467a59b1c28e433ee8bf.png"},{"id":80377190,"identity":"44fd517f-9835-4b96-83e5-728293d9e7d2","added_by":"auto","created_at":"2025-04-11 08:09:02","extension":"png","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":304842,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for CM\u003c/p\u003e","description":"","filename":"3.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/382f508fb399e3c0c951bcb5.png"},{"id":80377486,"identity":"1e2a6282-8a9a-4c91-8c23-1b5a17e76f6c","added_by":"auto","created_at":"2025-04-11 08:17:01","extension":"png","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":277561,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for GM\u003c/p\u003e","description":"","filename":"4.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/708d13484697e08bde2ff5a9.png"},{"id":80377184,"identity":"1394b7f7-3cd6-4142-b2b1-526fc89140ba","added_by":"auto","created_at":"2025-04-11 08:09:01","extension":"png","order_by":5,"title":"Figure 5","display":"","copyAsset":false,"role":"figure","size":232373,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for IM\u003c/p\u003e","description":"","filename":"5.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/d2bccd97c4bb942066b2acc1.png"},{"id":80377161,"identity":"21961850-3428-4fe7-9cfb-bbfd91188291","added_by":"auto","created_at":"2025-04-11 08:08:58","extension":"png","order_by":6,"title":"Figure 6","display":"","copyAsset":false,"role":"figure","size":250072,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for SD\u003c/p\u003e","description":"","filename":"6.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/d51d88c26c52a3a4665991e6.png"},{"id":80377165,"identity":"b845b9d8-1b6c-444e-bdb4-649a9354d810","added_by":"auto","created_at":"2025-04-11 08:08:58","extension":"png","order_by":7,"title":"Figure 7","display":"","copyAsset":false,"role":"figure","size":225557,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for SE\u003c/p\u003e","description":"","filename":"7.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/20c35c84fde6ad175216750b.png"},{"id":80377167,"identity":"43d009a5-782a-4732-9be1-c753f97da91d","added_by":"auto","created_at":"2025-04-11 08:08:58","extension":"png","order_by":8,"title":"Figure 8","display":"","copyAsset":false,"role":"figure","size":252178,"visible":true,"origin":"","legend":"\u003cp\u003eForest plot of the α\u003csub\u003e+\u003c/sub\u003e coefficients reported for total scale\u003c/p\u003e","description":"","filename":"8.png","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/2ca0daf85a83b27edd8a7bcb.png"},{"id":80378331,"identity":"88bb5e92-b166-4806-9b34-1fd03093e369","added_by":"auto","created_at":"2025-04-11 08:25:01","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1987689,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/c9b9a296-539a-48fd-908f-11938ebe7bf1.pdf"},{"id":80377185,"identity":"d883f7ba-f9fc-4779-952f-8692bd336af7","added_by":"auto","created_at":"2025-04-11 08:09:01","extension":"r","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":15681,"visible":true,"origin":"","legend":"\u003cp\u003eRcode_RG_SMQII\u003c/p\u003e","description":"","filename":"RGMeta.r","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/3bcb15c37ac2cc83686d9b14.r"},{"id":80377159,"identity":"32bb135b-f877-4d9d-9a1b-b27aa5120b75","added_by":"auto","created_at":"2025-04-11 08:08:57","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":102525,"visible":true,"origin":"","legend":"\u003cp\u003eSupplementary_RG_SMQ_II\u003c/p\u003e","description":"","filename":"SupplementaryDocumentsRGSMQII.docx","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/00997fc58354a44992cc5273.docx"},{"id":80377166,"identity":"0f7b8331-69fe-4470-9354-4d82c2467a19","added_by":"auto","created_at":"2025-04-11 08:08:58","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":1681000,"visible":true,"origin":"","legend":"","description":"","filename":"Tables.docx","url":"https://assets-eu.researchsquare.com/files/rs-6419938/v1/64613b5f00b84aea150181da.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003e\u003cstrong\u003eA Reliability Generalization Meta-Analysis of the Science Motivation Questionnaire II\u003c/strong\u003e\u003c/p\u003e","fulltext":[{"header":"Background","content":"\u003cp\u003eOne of the fundamental goals of science education is to cultivate scientifically literate citizens who have a solid understanding of scientific knowledge and can use evidence-based reasoning to make informed decisions about issues related to science and society (OECD, 2007; Roberts, \u003cspan class=\"CitationRef\"\u003e2007\u003c/span\u003e). However, recent surveys indicate that students of varying ages lack basic scientific literacy (Aguilera \u0026amp; Perales-Palacios, 2018). Researchers have highlighted a bidirectional relationship between the enhancement of scientific literacy and science motivation (Van Vo \u0026amp; Csap\u0026oacute;, \u003cspan class=\"CitationRef\"\u003e2022\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eGiven the important role of motivation to learn science, several studies have explored how to examine the level of motivation of college, secondary, and elementary students (Breland et al., 2023; Huda \u0026amp; Rohaeti, 2023; Jian-Xin et al., \u003cspan class=\"CitationRef\"\u003e2023\u003c/span\u003e; Meulenbroeks et al., \u003cspan class=\"CitationRef\"\u003e2024\u003c/span\u003e; Shin et al., \u003cspan class=\"CitationRef\"\u003e2023\u003c/span\u003e). Thus, researchers have utilized the Science Motivation Questionnaire-II (SMQ-II) developed by Glynn et al. (\u003cspan class=\"CitationRef\"\u003e2011\u003c/span\u003e) for different purposes. For example, the SMQ-II has been employed in conjunction with other scales to develop a model and identify factors for STEM career interest formation among students (Alexopoulos et al., 2021; Lei, 2024; Riccitelli, 2015), as well as to evaluate undergraduate students\u0026rsquo; perceived motivation and engagement in game theme-based learning (B\u0026oacute;nus et al., 2024; Moro et al., 2023; Watt, 2020).\u003c/p\u003e\n\u003cp\u003eOther studies have employed the SMQ-II to investigate motivation to learn specific science disciplines, such as chemistry and physics (Ardura \u0026amp; P\u0026eacute;rez-Bitri\u0026aacute;n, 2018; Chen et al., 2023; de Souza et al., 2022; Hibbard et al., 2016; Kahraman, 2023; Kwarikunda et al., 2021; Lee \u0026amp; Mun, 2023; Lei, 2024; Salta \u0026amp; Koulougliotis, 2015, 2020; Sch\u0026ouml;nfelder \u0026amp; Bogner, 2020). The instrument has been translated and adapted to different languages and cultural contexts to apply to Chinese (Dong et al., 2020), Spanish (Ardura \u0026amp; P\u0026eacute;rez-Bitri\u0026aacute;n, 2018), Greek (Salta \u0026amp; Koulougliotis, 2020), German (Schumm \u0026amp; Bogner, 2016), Korean (Rachmatullah \u0026amp; Ha, 2019), and Turkish (Tosun, \u003cspan class=\"CitationRef\"\u003e2013\u003c/span\u003e) students. Researchers have used the SMQ-II to assess students\u0026rsquo; motivation in a fourth-year technical aircraft systems course (Ng \u0026amp; Chu, \u003cspan class=\"CitationRef\"\u003e2021\u003c/span\u003e), measure dental students\u0026rsquo; oral medicine learning motivation in Indonesia, and investigate the learning motivation profile of Japanese pharmacy students (Rahmayanti et al., \u003cspan class=\"CitationRef\"\u003e2020\u003c/span\u003e). The scale was also used to determine how different motivations of learners relate to their engagement in a massive open online astronomy course on the Coursera platform (Formanek et al., 2019). Consequently, discipline-specific versions of the SMQ-II emerged as researchers adapted the instrument for their studies. For instance, the Chemistry Motivation Questionnaire II (CMQ-II) (Austin et al., 2018) and Physics Motivation Questionnaire II (PMQ-II) (Kwarikunda et al., 2020) are frequently used versions of the original questionnaire that replace the word \u0026ldquo;science\u0026rdquo; with \u0026ldquo;chemistry\u0026rdquo; or \u0026ldquo;physics,\u0026rdquo; respectively.\u003c/p\u003e\n\u003cp\u003eThe SMQ-II (Glynn et al., \u003cspan class=\"CitationRef\"\u003e2011\u003c/span\u003e) represents a revised version of an instrument presented in an earlier study by Glynn et al. (\u003cspan class=\"CitationRef\"\u003e2009\u003c/span\u003e). The first version of the scale consisted of 30 items in five subscales: intrinsic motivation and personal relevance (ten items), self-efficacy and assessment anxiety (nine items), self-determination (four items), career motivation (two items), and grade motivation (five items). Two years later, the researchers modified the questionnaire to improve the construct validity, based on the results of the first version of the questionnaire, focus group interviews, expert opinions, and a pilot study. Sixteen items remained unchanged in the revised version, and nine were added. As a result, the SMQ-II utilized in this meta-analysis consists of 25 items with five subscales: intrinsic motivation (IM), self-determination (SD), self-efficacy (SE), career motivation (CM), and grade motivation (GM). Five items in each subscale are scaled on a five-point Likert scale (from \u003cem\u003enever\u003c/em\u003e to \u003cem\u003ealways).\u003c/em\u003e In the original validation study, reliability was reported as 0.92 for the total score, 0.92 for CM, 0.81 for GM, 0.89 for IM, 0.88 for SD, and 0.83 for SE.\u003c/p\u003e\n\u003ch3\u003eReliability and Reliability Generalization (RG)\u003c/h3\u003e\n\u003cp\u003eReliability refers to the degree of consistency and stability of the scores from a measurement tool (Crocker \u0026amp; Algina, \u003cspan class=\"CitationRef\"\u003e1986\u003c/span\u003e). Errors in the measurement process reduce consistency; the source of error may vary from the scale itself, the sample (e.g., age, gender, motivation, etc.), the way the scale is applied, or the scoring complexity (Barnes et al., \u003cspan class=\"CitationRef\"\u003e2002\u003c/span\u003e; Dawis, \u003cspan class=\"CitationRef\"\u003e1987\u003c/span\u003e; Thompson \u0026amp; Vacha-Haase, \u003cspan class=\"CitationRef\"\u003e2000\u003c/span\u003e). It is critical to report reliability to support the reproducibility of research findings, particularly in the case of scores from a scale that is widely used in different languages and cultures, different sample groups, and different disciplines, such as the SMQ-II. The most appropriate option that allows a quantitative summary of reliability coefficients is the reliability generalization (RG) meta-analysis proposed by Vacha-Haase (\u003cspan class=\"CitationRef\"\u003e1998\u003c/span\u003e). RG studies are meta-analyses that calculate an average reliability estimate and can explain the heterogeneity of the coefficients (Henson \u0026amp; Thompson, \u003cspan class=\"CitationRef\"\u003e2002\u003c/span\u003e; Rodriguez \u0026amp; Maeda, \u003cspan class=\"CitationRef\"\u003e2006\u003c/span\u003e; S\u0026aacute;nchez-Meca et al., \u003cspan class=\"CitationRef\"\u003e2013\u003c/span\u003e; Vacha-Haase \u0026amp; Thompson, \u003cspan class=\"CitationRef\"\u003e2011\u003c/span\u003e). RG analysis also allows the examination of moderator variables that can explain a heterogeneity between the reported reliability coefficients (S\u0026aacute;nchez-Meca et al., \u003cspan class=\"CitationRef\"\u003e2013\u003c/span\u003e). In other words, RG meta-analysis aims to estimate the average reliability of test scores based on various test applications in different samples and contexts, as well as evaluating the consistency across test applications. In addition, if differences in reliability coefficients are observed, it allows the determination of study and sample characteristics that may explain these differences from a statistical perspective (Thompson, \u003cspan class=\"CitationRef\"\u003e2002\u003c/span\u003e; Vacha-Haase, \u003cspan class=\"CitationRef\"\u003e1998\u003c/span\u003e).\u003c/p\u003e\n\u003cp\u003eThe SMQ-II is a scale that exhibits strong psychometric properties and has become widely used internationally, after being adapted to more than ten languages to apply to various sample groups across cultures. Since the development of the SMQ-II, studies have been conducted on its construct validity (Glynn et al., \u003cspan class=\"CitationRef\"\u003e2011\u003c/span\u003e), measurement invariance for characteristics like gender and culture (Dong et al., 2020; Toma et al, 2023), construct validity in different languages and disciplines (Ardura \u0026amp; P\u0026eacute;rez-Bitri\u0026aacute;n, 2018; de Souza et al., 2022; Toma et al., 2023) and adaptation (Salta \u0026amp; Koulougliotis, 2015). However, to our knowledge, no researchers have conducted an RG meta-analysis of the scale. Therefore, this study aims to conduct an RG meta-analysis on the SMQ-II scale due to its widespread use in different languages and cultures, its translations and adaptations into many languages, and its frequent application in the literature. The purpose of this study was to (a) determine a precise estimate of the overall reliability coefficient for the SMQ-II; (b) investigate how reliability coefficients vary; (c) determine whether there is heterogeneity between reliability estimates, and if so, conduct moderator analyses to identify study characteristics that could explain the variability.\u003c/p\u003e"},{"header":"Method","content":"\u003ch2\u003eSelection Criteria\u003c/h2\u003e\u003cp\u003eTo improve its reporting, this paper follows the Reliability Generalization Meta Analysis (REGEMA), a 30-item checklist by Sánchez-Meca et al. (\u003cspan citationid=\"CR71\" class=\"CitationRef\"\u003e2021\u003c/span\u003e), which we provide as a supplement to this paper. First, a set of selection inclusion criteria was created to identify the studies for inclusion in the meta-analysis. Specifically we looked for studies that: (a) used the original SMQ-II (Glynn et al., \u003cspan citationid=\"CR29\" class=\"CitationRef\"\u003e2011\u003c/span\u003e) or any of its adapted versions (Ardura \u0026amp; Pérez-Bitrián, 2018; Salta \u0026amp; Koulougliotis, 2020), (b) were published or unpublished (e.g., Ph.D. dissertations) between 2011–2024, (c) were written in English, (d) allowed access to full text, (e) reported Cronbach’s alpha reliability coefficient for the total SMQ-II or any of its subscales, and (f) focused on the target population in STEM fields regardless of age (e.g., university students, high school students, etc.). We also set exclusion criteria, specifically excluding studies that: (a) employed the earlier version of the SMQ (Glynn et al., \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e2009\u003c/span\u003e), (b) reported alpha values only from the original scale or from previous studies, (c) reported a reliability coefficient other than Cronbach’s alpha, (d) were book chapters or conference proceedings, (e) added different items to total SMQ-II or any of its subscales, (f) focused on a field other than science education.\u003c/p\u003e\u003ch3\u003eSearch Strategies\u003c/h3\u003e\u003cp\u003eArticles were obtained for the study sample by querying the Web of Science, ERIC, Scopus, and ProQuest databases with the search terms \u003cem\u003eScience Motivation Questionnaire II\u003c/em\u003e and \u003cem\u003eSMQ II\u003c/em\u003e. The search was limited to articles published in or after 2011, the when the SMQ-II scale was first used. We ensured search consistency across the research team by first searching for a certain number of articles together, before completing separate searches in different databases. After the random assignment of electronic databases to four authors, a folder was created for each database. The abstracts of the selected publications were read and evaluated, and if they met the criteria, the full text was examined. All related documents were collected in a main file and duplicated entries were easily detected and deleted.\u003c/p\u003e\u003ch3\u003eData Extraction and Procedure\u003c/h3\u003e\u003cp\u003eA coding form was created following the RG meta-analysis literature as suggested by Henson and Thompson (\u003cspan citationid=\"CR35\" class=\"CitationRef\"\u003e2002\u003c/span\u003e), and the nature of the test was taken into consideration. We extracted Cronbach’s alpha for the SMQ-II from the studies that reported at least one reliability estimate for the total score or subscale scores along with the following potential moderators: (a) study type (\u003cem\u003earticle\u003c/em\u003e or \u003cem\u003edissertation/thesis)\u003c/em\u003e; (b) language of the SMQ-II utilized (\u003cem\u003eEnglish\u003c/em\u003e or \u003cem\u003eother\u003c/em\u003e); (c) study sample (\u003cem\u003ecollege students\u003c/em\u003e or \u003cem\u003eother\u003c/em\u003e); (d) test version (\u003cem\u003eoriginal\u003c/em\u003e or \u003cem\u003eadapted for a discipline\u003c/em\u003e); (e) sample size; and (f) gender distribution (percentage of females).\u003c/p\u003e\u003cp\u003eThe language of the SMQ-II utilized was coded into two categories. The studies in the dataset adapted the SMQ-II scale into twelve different languages; however, there was not enough data for each language category. Therefore, if the SMQ-II was delivered in English, the language moderator was coded as \u003cem\u003eEnglish\u003c/em\u003e or otherwise coded as \u003cem\u003eother\u003c/em\u003e. Due to a similar lack of data for each category in the data set, the test version was also coded into two categories. If the scale was presented in its original form as developed by Glyn et al. (2009) where the word \u003cem\u003escience\u003c/em\u003e was used, the test version moderator was coded as \u003cem\u003eoriginal\u003c/em\u003e; if it was adapted to another discipline (physics, chemistry, etc.), it was coded as \u003cem\u003eadapted\u003c/em\u003e.\u003c/p\u003e\u003cp\u003eIn some studies, more than one Cronbach alpha value was reported for more than one sample (experimental-control groups). If the reported values ​​were calculated separately for different samples, they were coded separately. However, other studies reported more than one alpha value for the same sample (Ardura \u0026amp; Pérez-Bitrián, 2018; Salta \u0026amp; Koulougliotis, 2015). In these studies, although alpha values were ​​calculated separately for different groups, there was also an alpha value for the combined sample where the groups were merged. In this case, only the alpha values ​​for each group were included in the analysis. Some studies also added or removed items from the total or subdimensions of the scale (Park \u0026amp; ​​Kim, 2021; Zhdanov et al., 2022). The valid item numbers and reliability values ​​were coded in case of item removal. However, if new items were added to the scale, these data were not included in the analysis.\u003c/p\u003e\u003cp\u003eSearching for SMQ-II from 2011–2024 yielded a total of 259 studies, 76 of which were duplicate publications. Of the 183 studies included in the full-text review, 78 studies were excluded from the analysis because they did not meet the eligibility criteria: they utilized SMQ rather than SMQ-II, reported prior Cronbach’s alpha values rather than computing, were not written in English, or conducted research in disciplines other than science (e.g., aviation, medicine). Thus, 49 studies that met the inclusion criteria were included in the analysis. The REGEMA flow chart in Fig.\u0026nbsp;\u003cspan refid=\"Fig1\" class=\"InternalRef\"\u003e1\u003c/span\u003e presents the selection process details.\u003c/p\u003e\u003cp\u003eReliability induction is a type of publication bias specific to RG meta-analysis, involving reporting a reliability coefficient from prior research but not for the study itself (Vacha-Haase et al., \u003cspan citationid=\"CR84\" class=\"CitationRef\"\u003e2000\u003c/span\u003e). In our study, 30 of the 183 eligible studies did not report the reliability value, and nine reported the reliability value from the original SMQ-II study, resulting in a reliability induction rate of 21.31%. The reliability induction rate was obtained as an average of 78.6% in a systematic study of 100 RG meta-analyses, including more than 40,000 empirical studies in psychology (Sánchez-Meca et al., 2015). Hence, it can be stated that our study’s reliability induction rate was at a reasonable level.\u003c/p\u003e\u003ch3\u003eIntercoder Agreement\u003c/h3\u003e\u003cp\u003eThree members of the research team coded the studies independently after a consensus meeting where three different studies were examined and discussed together. Then, the coders coded randomly selected ten studies that reported 60 reliability coefficients, to calculate Krippendorf’s Alpha coefficient for intercoder agreement. This analysis was performed using the \u003cem\u003eirr\u003c/em\u003e package (Gamer et al., \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e2019\u003c/span\u003e) in the R statistical software (R Core Team, \u003cspan citationid=\"CR62\" class=\"CitationRef\"\u003e2025\u003c/span\u003e). The intercoder reliability coefficient for the three coders was 0.99, indicating near-perfect agreement.\u003c/p\u003e\u003ch2\u003eData Analysis\u003c/h2\u003e\u003cp\u003eSimilar to Sandoval-Lentisco et al. (\u003cspan citationid=\"CR72\" class=\"CitationRef\"\u003e2023\u003c/span\u003e), we conducted separate RG meta-analyses for each SMQ-II subscale before analyzing total scores. A random effects model was utilized, as we assumed that the included reliability coefficients were a random sample from the field. All statistical analyses were conducted using the \u003cem\u003emetafor\u003c/em\u003e package in R (Viechtbauer, \u003cspan citationid=\"CR87\" class=\"CitationRef\"\u003e2010\u003c/span\u003e). Beretvas and Pastor (\u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2003\u003c/span\u003e) suggest that alpha values reported in studies using the specified scale should not be used directly because they are usually negatively skewed. Therefore, we conducted the analyses by transforming the alpha values. We first computed outcome measures and sampling variances based on Bonnet’s transformation (2002) for the alpha values using -ln(1 - α\u003csub\u003ei\u003c/sub\u003e), where α\u003csub\u003ei\u003c/sub\u003e is the reported reliability coefficient. We then employed the random effects model with the restricted maximum likelihood (REML) and back-transformed the results using \u003cem\u003etransf = transf.iabt\u003c/em\u003e argument to compute 1–1/e\u003csup\u003e(estimate)\u003c/sup\u003e and reported point estimates and confidence intervals.\u003c/p\u003e\u003cp\u003eWe examined \u003cem\u003eQ\u003c/em\u003e statistics and \u003cem\u003eI\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e values ​to determine the heterogeneity between alpha coefficients. Statistically significant \u003cem\u003eQ\u003c/em\u003e statistics (Cochran, \u003cspan citationid=\"CR15\" class=\"CitationRef\"\u003e1954\u003c/span\u003e) and large \u003cem\u003eI\u003c/em\u003e\u003csup\u003e\u003cem\u003e2\u003c/em\u003e\u003c/sup\u003e values ​​(\u0026gt; 75%) are accepted as an indication of heterogeneity (Higgins et al., \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e2003\u003c/span\u003e). To determine the reasonable reliability estimate interval of the studies, 95% confidence intervals were calculated, and forest plots containing these confidence intervals were created to provide a visual inspection of the effect sizes of the studies. For categorical and continuous moderator variables, heterogeneity analyses were performed using univariate meta-regression models in the \u003cem\u003emetafor\u003c/em\u003e package.\u003c/p\u003e\u003cp\u003eThis RG meta-analysis was conducted with 49 studies: 40 articles (81.63%) and 9 theses (18.37%). The studies were conducted in over 20 countries, including the USA (the majority at 36.73%), Germany, Indonesia, Brazil, China, Greece, and Turkey. Thirty-six studies applied the SMQ-II scale in English (73.47%), while 13 studies applied it in eleven different languages (26.53%), including Portuguese, Greek, German, Chinese, Spanish, Russian, and Turkish. While 44.90% of the studies involved university students, 55.10% of the sample consisted of students at lower educational levels. The studies obtained data from 21,046 people in total. Since not all the included studies reported reliability coefficients for both the total score and for each subscale score, the number of reliability coefficients in each model varied, as explained in the results section.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003ePublication Bias\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe utilized funnel plots, Egger regression test (Egger et al., 1997), trim and fill method, and RG\u0026nbsp;fail-safe N (Howell \u0026amp; Shields, 2008) analysis to examine publication bias for the coefficients of SMQ-II total scores. Egger\u0026rsquo;s regression test showed a statistically significant asymmetry (\u003cem\u003et\u003c/em\u003e=-3.530, sd=24, \u003cem\u003ep\u003c/em\u003e=.002), possibly due to a small study effect or possible publication bias. However, the trim and fill method analysis determined that there were no missing studies on the right side (see supplementary). The funnel plot in Figure 2 shows a symmetrical distribution, but with some asymmetry due to points remaining outside in the upper right region. Following Howell and Shields (2008), we first computed the lower bound estimate of unweighted alpha for the total score in 26 included studies. The average alpha for these 26 studies was .93 with a standard deviation of .03, with a plausible worst-case reliability coefficient of .80 standard deviations below the average. For the 134 eligible but not useful studies, average alpha was estimated as .906, hence resulting in a lower bound estimate of .91. Again, following Howell and Shields (2008) with similar criteria and a threshold reliability of .80, the RG fail-safe N was computed as 39. In other words, the mean population reliability of SMQ-II scores would be less than .80 if there were 39 file drawer studies with an unweighted average reliability of .712, a value far below the estimated lower bound. Overall, the findings obtained in the following RG meta-analysis are expected to be highly robust to publication bias.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eMean Reliability and Heterogeneity\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTable 1 reports the year, type, and country for the included studies. Forty of the studies were articles, whereas nine were theses. Although the studies conducted in more than 20 countries, most were conducted in the USA (36.73%).\u003c/p\u003e\n\u003cp\u003eFor the subscales, 47 alpha values were obtained from 32 different studies for CM, 45 from 33 studies for GM, 43 from 32 studies for IM, 46 from 31 studies for SD, and 50 from 34 studies for SE. For the SMQ-II total score, 27 values were obtained from 23 different studies. The data analyzed had been collected from 12,434 individuals for the CM subscale, 12,683 for GM, 16,211 for IM, 11,559 for SD, 13,675 for SE, and 6,881 individuals from more than three countries for the total scale. Figures 3\u0026ndash;8 present forest plots of reliability coefficients for the overall scale and each subscale.\u003c/p\u003e\n\u003cp\u003eThe total scale reliability values of the studies ranged from .85 (Moro \u0026amp; Billote, 2023) to .97 (Huda \u0026amp; Rohaeti, 2023; Lee \u0026amp; Mun, 2023; Razali et al., 2020). The forest plot in Figure 8 shows that the weights of all studies are close to each other, apart from a few studies (Lei, 2024; Migalang \u0026amp; Azuelo, 2020; Watt, 2020).\u003c/p\u003e\n\u003cp\u003eTable 2 shows the main results for the total and each of the five subscales of the SMQ-II scale. The weighted average calculated for alpha for the total score of the SMQ-II scale was 0.94 (95% CI[.93, .95]) and was statistically significant at the \u003cem\u003ep\u003c/em\u003e\u0026lt;.001 level. The \u003cem\u003eQ\u003c/em\u003e statistic was significant (\u003cem\u003eQ\u003c/em\u003e=664.0269, \u003cem\u003ep\u003c/em\u003e\u0026lt;.001) and the \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e statistic was 96.36.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAmong the subscales of the SMQ-II, \u0026alpha;\u003csub\u003e+\u003c/sub\u003e=0.88 (95% CI:[0.86, 0.90]) for CM, \u0026alpha;\u003csub\u003e+\u003c/sub\u003e=0.83 (95% CI:[0.80, 0.85]) for GM, \u0026alpha;\u003csub\u003e+\u003c/sub\u003e=0.85 (95% CI:[0.82, 0. 87]), \u0026alpha;\u003csub\u003e+\u003c/sub\u003e=0.84 (95% CI:[0.82, 0.86]) for SD, and \u0026alpha;\u003csub\u003e+\u003c/sub\u003e=0.85 (95% CI:[0.84, 0.87]) for SE were statistically significant at \u003cem\u003ep\u003c/em\u003e\u0026lt;.001 level. Cochran\u0026rsquo;s \u003cem\u003eQ\u003c/em\u003e statistic and \u003cem\u003eI\u003csup\u003e2\u0026nbsp;\u003c/sup\u003e\u003c/em\u003estatistics were found to be statistically significant in the CM (\u003cem\u003eQ\u003c/em\u003e=1436.1580, \u003cem\u003ep\u003c/em\u003e\u0026lt;.0001, \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=96.96), GM (\u003cem\u003eQ\u003c/em\u003e=1960.1004, \u003cem\u003ep\u003c/em\u003e\u0026lt;.0001, \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=96.98); IM (\u003cem\u003eQ\u003c/em\u003e=1412. 9436, \u003cem\u003ep\u003c/em\u003e\u0026lt;.0001, \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=97.40), SD (\u003cem\u003eQ\u003c/em\u003e=1059.1763, \u003cem\u003ep\u003c/em\u003e\u0026lt;.0001, \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=95.88), and SE subscales (\u003cem\u003eQ\u003c/em\u003e=1366.1194, \u003cem\u003ep\u003c/em\u003e\u0026lt;.0001, \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=95.95). Accordingly, the statistically significant \u003cem\u003eQ\u003c/em\u003e statistics and high \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e values indicate a high level of heterogeneity among the studies reporting reliability values for the subscales of the SMQ-II scale.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eModerator Analysis\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe \u003cem\u003eQ\u003c/em\u003e statistics and high \u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e values (\u0026gt;92) show significant heterogeneity between the mean alpha coefficients for the SMQ-II total score and its subscales, as supported by the forest plots. Moderator analyses were conducted to investigate potential sources of heterogeneity, with study type, study language, sample type, and test version as categorical variables, and sample size and female percentage as continuous variables. While there were missing values for continuous moderator variables in the data set, no imputation was made for these values, and they were considered missing data.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe results of\u0026nbsp;the meta-regression analysis for categorical moderator variables are presented in Table 3. The results show that there are no significant differences in the reliability coefficients of moderator variables in the CM, GM, and IM subscales (\u003cem\u003ep\u003c/em\u003e\u0026gt;.05). However, the test version was a statistically significant moderator variable in the SD (Q\u003csub\u003eM\u003c/sub\u003e=7.712, \u003cem\u003ep\u003c/em\u003e=.006) and SE (Q\u003csub\u003eM\u003c/sub\u003e=9.726, \u003cem\u003ep\u003c/em\u003e=.002) subscales. The results of the analysis revealed that the average reliability coefficients of the original test were higher than those of the adapted test. In addition, the sample type was a statistically significant moderator variable for the total scale (Q\u003csub\u003eM\u003c/sub\u003e=10.540, \u003cem\u003ep\u003c/em\u003e=.001). The average reliability value calculated for the samples consisting of university students (0.92) was lower than the samples consisting of other students (0.95). Table 3 presents the average reliability coefficients of the moderator variables for the total scale and subscales.\u003c/p\u003e\n\u003cp\u003eTable 4 presents the results of the meta-regression analysis of continuous moderator variables.\u0026nbsp;The sample size variable had a statistically significant effect on the total scale (\u003cem\u003eb\u003csub\u003ej\u003c/sub\u003e\u003c/em\u003e=.001, SE=-.000, \u003cem\u003ep\u003c/em\u003e=.028, \u003cem\u003eR\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=13.87). Although this finding indicates that increasing the sample size significantly affects the reliability coefficient calculated for the total scale, it also indicates that the effect is relatively small. However, the sample size does not have a statistically significant effect on the reliability coefficient of any of the subscales (p\u0026gt;.05).\u003c/p\u003e\n\u003cp\u003eGender distribution, one of the moderator variables, shows a statistically significant negative effect for GM (\u003cem\u003eb\u003csub\u003ej\u003c/sub\u003e\u003c/em\u003e=-.011, SE=.005, \u003cem\u003ep\u003c/em\u003e=.028, \u003cem\u003eR\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e= 12.08). This finding indicates that an increase in the proportion of female participants has a significant, but relatively small effect on the reliability coefficient for the GM subscale. For the total scale and other subscales (CM, IM, SD, and SE), gender distribution had no significant effect (\u003cem\u003ep\u003c/em\u003e\u0026gt;.05).\u003c/p\u003e"},{"header":" Conclusions and Discussion","content":"\u003cp\u003eThis reliability generalization meta-analysis study aimed to estimate the mean reliability coefficient of the SMQ-II scale and its subscales and examine moderator variables that may cause variability between studies. For this purpose, a meta-analysis was conducted on 49 studies obtained within the scope of the criteria determined. The mean reliability coefficient for the SMQ-II total scale score was calculated as 0.94. This value is very close to the original reliability coefficient of 0.92 reported by Glynn et al. (2011), but still slightly higher. The mean reliability coefficients of the SMQ-II subscales were between 0.83–0.88, which can be expressed as “very good” according to DeVellis (2003). When compared with the original reliability coefficients of the scale, the mean reliability coefficient was higher in two subscales (GM and SE) and lower in three subscales (CM, IM, and SD). A large heterogeneity (\u003cem\u003eI\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e=96.36) was found between the coefficients for the SMQ-II total scale. Thus, we recommend that researchers who plan to use the scale calculate the reliability value for their own samples and not make reliability induction.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eOur results also showed that the reliability values for the total scale varied according to the type of sample (university versus lower-level students). The mean reliability value calculated for samples consisting of university students (0.92) was lower than that for samples consisting of lower-level students (0.95). While the scientific motivation of university students is related to individual interest and academic/career goals, it may depend on teacher or parental guidance for students at other educational levels. Second, students’ levels of intellectual and cognitive development may affected their responses to the scale. Less advanced students may have perceived and made sense of abstract concepts such as scientific motivation differently than more advanced students. The result that the sample type affects the average reliability estimate mirrors the findings of previous studies by Vacha-Haase (1998), Yin and Fan (2000), Eser and Dogan (2023), and Batı and Irmak (2024).\u003c/p\u003e\n\u003cp\u003eAnother important moderator variable for total score reliability was sample size. An increase in sample size significantly affected the reliability coefficient, but this effect was statistically expected, as large samples provide more stable results than small samples. The positive relationship between sample size and reliability coefficients has yielded similar results in other studies (Sandoval-Lentisco et al., 2023).\u003c/p\u003e\n\u003cp\u003eSignificant heterogeneity was also found between the reliability coefficients in the subscales. The moderator analyses to gauge heterogeneity concluded that the type of study, language, sample size, and sample type variables did not affect the subscale reliability values; however, test version, one of the moderator variables, did impact the change of reliability values in the SD and SE subscales. Moreover, the average reliability coefficients of the original test version were higher than the adapted test version.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAnother moderator variable, female percentage, was found to have an effect on the reliability values in the GM subscale. However, this effect was relatively small. This result could stem from the differing levels of gender equality in the countries where the studies in the sample were conducted. Earlier measurement invariance studies for the SMQ-II did not reveal significant differences between genders (Dong et al., 2020; Toma et al., 2023). However, our results suggest that new studies should employ measurement invariance studies to assure that the SMQ-II measures similarly across genders.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe present study’s analysis of research conducted on individuals from more than 15 countries, with sample sizes ranging from 6,800 to 16,000, revealed very high reliability coefficients for both the total and subscale scores of the SMQ-II. However, reliability is not a fixed characteristic of scales, rather a characteristic of the scores obtained at the end of the measurements (Crocker \u0026amp; Algina, 1986). Organizations such as the APA (Wilkinson, 1999), American Educational Research Association, and National Council on Measurement in Education recommend that researchers calculate and report the reliability values of their own studies. The literature suggests that many researchers do heed this recommendation and present the scale either by referring to the original reliability coefficient or the reliability coefficient obtained in another study, which Vacha-Haase et al. (2000) refer to as “reliability induction.” Therefore, it is important for researchers to report the reliability coefficients obtained from their own studies, regardless of which scale they use. Investigations have shown that researchers’ most commonly reported reliability coefficient is Cronbach’s alpha (Scherer \u0026amp; Teo, 2020). There are several potential reasons for this. First, Cronbach’s alpha is easier to calculate than other reliability coefficients and can be calculated directly through many statistical programs. Second, researchers may prefer it because it can be calculated without multiple test administrations (Scherer \u0026amp; Teo, 2020). Third, calculating other reliability coefficients may require more complex statistical knowledge and computational skills. It may also be that these methods are not well known by researchers, or that academic journal reviewers and editors are more familiar with Cronbach’s alpha and demand this coefficient. Therefore, researchers may be inclined to report Cronbach’s alpha reliability coefficient to minimize potential difficulties in the publication process.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eImplications and\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eLópez-Ibáñez et al. (2024) analyzed the RG meta-analysis studies conducted between 1998–2020 and reported that Cronbach’s alpha was dominant among the reliability coefficients used, despite criticisms (Yang \u0026amp; Green, 2011). In this context, it is recommended that researchers turn to alternative reliability values and report these values along with Cronbach’s alpha to evaluate reliability.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn our RG meta-analysis of the SMQ-II, the type of study, study language, sample type, test version, sample size, and female percentage were identified as moderator variables to determine the sources of heterogeneity. Future studies can focus on different moderator variables to investigate the sources of heterogeneity. More in-depth analyses are needed, especially for sample types that show statistically significant differences across sources of heterogeneity. Future research could focus on systematically examining how reliability coefficients vary across different age groups, educational levels, and cultural contexts. Such studies would make important contributions to assessing the applicability of the scales across different demographic and socio-cultural groups.\u003c/p\u003e\n\u003cp\u003eThe SMQ-II has been translated into many languages and modified for use in a variety of fields, including biology, chemistry, physics, and aviation. However, our study only considered English-language research, which poses another limitation since it results in the exclusion of research in other languages. At the same time, although there are adaptation studies in many disciplines, the test version variable of the scale could not be examined separately according to these disciplines due to the limited number of studies in the data set. In our study, the studies using the original form of the scale were coded as “original” in the test version, and the versions adapted to different disciplines were coded as “adapted” and included in the analysis. In this context, the inability to conduct detailed analyses on a discipline-by-discipline basis regarding the translation and adaptation processes of the scale also creates a limitation. Since we\u0026nbsp;only reviewed studies that applied the SMQ-II to science education, future RG meta-analyses of the scale should include studies in different disciplines (e.g., aviation or medicine) and languages. Detailed reporting of different reliability coefficients, sample characteristics, and descriptive statistics of the sampled studies would allow future research to focus on new coefficients and moderator variables.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTechnological advances and digital learning environments affect student motivation towards science and may consequently change how students respond to tests and questionnaires. In this context, differences can be observed between the reliability coefficients obtained from SMQ-II assessments applied in traditional paper-and-pencil form and the coefficients obtained from digital assessments. However, there is not enough data in the studies examined in the present research to evaluate this difference. We recommend that this situation be systematically addressed in future RG meta-analysis studies. Despite the aforementioned limitations, it is still reasonable to assert that the SMQ-II scale has strong internal consistency and robustness to adaptations for related disciplines.\u0026nbsp;\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n \u003cli\u003eAguilera, D., \u0026amp; Perales-Palacios, F. (2020). What effects do didactic interventions have on students\u0026rsquo; attitudes towards science? A Meta-Analysis. \u003cem\u003eResearch in Science Education,\u003c/em\u003e 50, 573\u0026ndash;597. https://doi.org/10.1007/s11165-018-9702-2\u003c/li\u003e\n \u003cli\u003e*Alexopoulos, A. N., Paolucci, P., Sotiriou, S. A., Bogner, F. X., Dorigo, T., Fedi, M., ... \u0026amp; Scianitti, F. (2021). The colours of the Higgs boson: a study in creativity and science motivation among high-school students in Italy. \u003cem\u003eSmart Learning Environments\u003c/em\u003e, \u003cem\u003e8\u003c/em\u003e, 1\u0026ndash;23. https://doi.org/10.1186/s40561-021-00169-4\u003c/li\u003e\n \u003cli\u003e*Ardura, D., \u0026amp; P\u0026eacute;rez-Bitri\u0026aacute;n, A. (2018). The effect of motivation on the choice of chemistry in secondary schools: adaptation and validation of the Science Motivation Questionnaire II to Spanish students. \u003cem\u003eChemistry Education Research and Practice\u003c/em\u003e, \u003cem\u003e19\u003c/em\u003e(3), 905\u0026ndash;918. https://doi.org/10.1039/C8RP00098K\u003c/li\u003e\n \u003cli\u003e*Austin, A. C., Hammond, N. B., Barrows, N., Gould, D. L., \u0026amp; Gould, I. R. (2018). Relating motivation and student outcomes in general organic chemistry. \u003cem\u003eChemistry Education Research and Practice\u003c/em\u003e, \u003cem\u003e19\u003c/em\u003e(1), 331\u0026ndash;341. https://doi.org/10.1039/C7RP00182G\u003c/li\u003e\n \u003cli\u003eBarnes, L. L., Harp, D., \u0026amp; Jung, W. S. (2002). Reliability generalization of scores on the Spielberger state-trait anxiety inventory. \u003cem\u003eEducational and psychological measurement\u003c/em\u003e, \u003cem\u003e62\u003c/em\u003e(4), 603\u0026ndash;618. https://doi.org/10.1177/0013164402062004005\u003c/li\u003e\n \u003cli\u003eBatı, K., \u0026amp; Irmak, Ş. (2024). A reliability generalization study of the STEM-CIS scale: Exploring moderator effects. \u003cem\u003eCurrent Psychology,\u003c/em\u003e 43\u003cstrong\u003e,\u003c/strong\u003e 35453\u0026ndash;35470. https://doi.org/10.1007/s12144-024-07036-8\u003c/li\u003e\n \u003cli\u003eBeretvas, S. N., \u0026amp; Pastor, D. A. (2003). Using mixed-effects models in reliability generalization studies. \u003cem\u003eEducational and Psychological Measurement\u003c/em\u003e, \u003cem\u003e63\u003c/em\u003e(1),75\u0026ndash;95.https://doi.org/10.1177/0013164402239318\u003c/li\u003e\n \u003cli\u003e*Bernard, R. A. (2018). \u003cem\u003eThe relationship of formal reasoning, motivation, and conceptual change: a quantitative study of introductory biology students across the United States\u003c/em\u003e (Doctoral dissertation, Montana State University).\u003c/li\u003e\n \u003cli\u003eBonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. \u003cem\u003eJournal of Educational and Behavioral Statistics\u003c/em\u003e, \u003cem\u003e27\u003c/em\u003e(4), 335\u0026ndash;340. https://doi.org/10.3102/10769986027004335\u003c/li\u003e\n \u003cli\u003e*B\u0026oacute;nus, L., Antal, E., \u0026amp; Korom, E. (2024). Digital game-based inquiry learning to improve eighth graders\u0026rsquo; inquiry skills in biology. \u003cem\u003eJournal of Science Education and Technology\u003c/em\u003e, 1\u0026ndash;17. https://doi.org/10.1007/s10956-024-10096-x\u003c/li\u003e\n \u003cli\u003e*Breland, H., Clark, C. M., Shaked, S., \u0026amp; Paquette-Smith, M. (2023). The benefits of participating in a learning assistant program on the metacognitive awareness and motivation of learning assistants. \u003cem\u003eCBE\u0026mdash;Life Sciences Education\u003c/em\u003e, \u003cem\u003e22\u003c/em\u003e(3), ar30. https://doi.org/10.1187/cbe.22-08-0156\u003c/li\u003e\n \u003cli\u003e*Chen, Y. C., Terada, T., \u0026amp; Aguirre-Mendez, C. (2023). Action research to explore argumentative writing projects supported with online simulation for developing conceptual knowledge and motivation to learn. \u003cem\u003eReading and Writing\u003c/em\u003e, \u003cem\u003e36\u003c/em\u003e(9), 2275\u0026ndash;2317. https://doi.org/10.1007/s11145-022-10363-6\u003c/li\u003e\n \u003cli\u003e*Christiansen, M. D. (2015). \u003cem\u003eScience\u0026nbsp;\u003c/em\u003e\u003cem\u003eas a direct measure of academic hope: the validation of the science hope scale\u003c/em\u003e. (Doctoral dissertation, Hofstra University).\u003c/li\u003e\n \u003cli\u003e*Chumbley, S., Hainline, M. S., Wells, T., \u0026amp; Haynes, J. C. (2022). Students\u0026apos; motivation to learn science through undergraduate-level agricultural coursework. \u003cem\u003eJournal of Agricultural Education\u003c/em\u003e, \u003cem\u003e63\u003c/em\u003e(1), 182\u0026ndash;199. https://doi.org/10.5032/jae.2022.01182\u003c/li\u003e\n \u003cli\u003eCochran, W. G. (1954). The combination of estimates from different experiments. \u003cem\u003eBiometrics\u003c/em\u003e, \u003cem\u003e10\u003c/em\u003e(1), 101\u0026ndash;129.\u003c/li\u003e\n \u003cli\u003e*Conner, L. C. (2021). \u003cem\u003eImpact of flipped learning on high school students\u0026rsquo; motivation, cognitive load, and perceptions of flipped approach for learning science\u003c/em\u003e (Doctoral dissertation, University of South Carolina).\u003c/li\u003e\n \u003cli\u003eCrocker, L., \u0026amp; Algina, J. (1986). \u003cem\u003eIntroduction to classical and modern test theory\u003c/em\u003e. Holt, Rinehart and Winston. Orlando, FL.\u003c/li\u003e\n \u003cli\u003eDawis, R. V. (1987).\u003cem\u003e\u0026nbsp;\u003c/em\u003eScale construction. \u003cem\u003eJournal of Counseling Psychology, 34,\u0026nbsp;\u003c/em\u003e481\u0026ndash;489.\u003c/li\u003e\n \u003cli\u003e*de Souza, R. T., da Silva, M. D. S. B., Barbato, D. M., de Guzzi, M. E. R., \u0026amp; Kasseboehmer, A. C. (2022). Motivation to learn chemistry: a thorough analysis of the CMQ-II within the Brazilian context. \u003cem\u003eChemistry Education Research and Practice\u003c/em\u003e, \u003cem\u003e23\u003c/em\u003e(4), 799\u0026ndash;810. https://doi.org/10.1039/D2RP00107A\u003c/li\u003e\n \u003cli\u003eDeVellis, R. F. (2003). Scale Development: Theory and Application. Thousand Oaks, CA: Sage.\u003c/li\u003e\n \u003cli\u003e*Dixon, K. L. (2017). \u003cem\u003eThe effect of the flipped classroom on urban high school students\u0026apos; motivation and academic achievement in a high school science course\u003c/em\u003e (Doctoral dissertation, Liberty University).\u003c/li\u003e\n \u003cli\u003e*Dixon, K., \u0026amp; Wendt, J. L. (2021). Science motivation and achievement among minority urban high school students: An examination of the flipped classroom model. \u003cem\u003eJournal of Science Education and Technology\u003c/em\u003e, \u003cem\u003e30\u003c/em\u003e(5), 642\u0026ndash;657. https://doi.org/10.1007/s10956-021-09909-0\u003c/li\u003e\n \u003cli\u003e*Dong, Z., Li, M., Minstrell, J., \u0026amp; Cui, Y. (2020). Psychometric properties of Science Motivation Questionnaire II‐Chinese version in two waves of longitudinal data. \u003cem\u003ePsychology in the Schools\u003c/em\u003e, \u003cem\u003e57\u003c/em\u003e(8), 1240\u0026ndash;1256. https://doi.org/10.1002/pits.22370\u003c/li\u003e\n \u003cli\u003eEgger, M., Smith, G. D., Schneider, M., \u0026amp; Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. \u003cem\u003ebmj\u003c/em\u003e, \u003cem\u003e315\u003c/em\u003e(7109), 629\u0026ndash;634.\u003c/li\u003e\n \u003cli\u003eEser, M. T., \u0026amp; Doğan, N. (2023). Life Satisfaction Scale: A meta-analytic reliability generalization study in Turkey sample. \u003cem\u003eTurkish Psychological Counseling and Guidance Journal\u003c/em\u003e, \u003cem\u003e13\u003c/em\u003e(69), 224\u0026ndash;239. https://doi.org/10.17066/tpdrd.1223320mn\u003c/li\u003e\n \u003cli\u003e*Formanek, M., Buxner, S., Impey, C., \u0026amp; Wenger, M. (2019). Relationship between learners\u0026rsquo; motivation and course engagement in an astronomy massive open online course. \u003cem\u003ePhysical Review Physics Education Research\u003c/em\u003e, \u003cem\u003e15\u003c/em\u003e(2), 020140. https://doi.org/10.1103/PhysRevPhysEducRes.15.020140\u003c/li\u003e\n \u003cli\u003eGamer, M., Lemon, J., Fellows, I., \u0026amp; Singh, P. (2019). \u003cem\u003eirr: Various coefficients of interrater reliability and agreement\u003c/em\u003e (Version 0.84.1) [R package]. Comprehensive R Archive Network. https://CRAN.R-project.org/package=irr\u003c/li\u003e\n \u003cli\u003eGlynn, S. M., Taasoobshirazi, G., \u0026amp; Brickman, P. (2009). Science motivation questionnaire: Construct validation with nonscience majors. \u003cem\u003eJournal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching\u003c/em\u003e, \u003cem\u003e46\u003c/em\u003e(2), 127\u0026ndash;146. https://doi.org/10.1002/tea.20267\u003c/li\u003e\n \u003cli\u003eGlynn, S. M., Brickman, P., Armstrong, N., \u0026amp; Taasoobshirazi, G. (2011). Science motivation questionnaire II: Validation with science majors and nonscience majors. \u003cem\u003eJournal of research in science teaching\u003c/em\u003e, \u003cem\u003e48\u003c/em\u003e(10), 1159\u0026ndash;1176. https://doi.org/10.1002/tea.20442\u003c/li\u003e\n \u003cli\u003e*Gomes de Freitas, R. T., Tolentino Cicuto, C. A., \u0026amp; Pazinato, M. S. (2020). Motivation to learn natural sciences in the context of a training of teacher\u0026rsquo;s course. \u003cem\u003eProblems of Education in the 21st Century\u003c/em\u003e, \u003cem\u003e78\u003c/em\u003e(3), 359\u0026ndash;370. http://dx.doi.org/10.33225/pec/20.78.359\u003c/li\u003e\n \u003cli\u003eJian-Xin, Y., Yu-Xuan, X., Tian, L., Chu-Fan, D., Yu-Ying, G., \u0026amp; Fortus, D. (2023). Disciplinary learning motivation and its external influencing factors: Taking physics in a \u0026ldquo;selection crisis\u0026rdquo; as an example. \u003cem\u003eResearch in Science Education\u003c/em\u003e, \u003cem\u003e53\u003c/em\u003e(4), 823\u0026ndash;839. https://doi.org/10.1007/s11165-023-10112-x\u003c/li\u003e\n \u003cli\u003eHakstian, A. R., \u0026amp; Whalen, T. E. (1976). A k-sample significance test for independent alpha coefficients. \u003cem\u003ePsychometrika\u003c/em\u003e, \u003cem\u003e41\u003c/em\u003e(2), 219\u0026ndash;231.\u003c/li\u003e\n \u003cli\u003e*Haviz, M., Maris, I. M., Azis, D., \u0026amp; Helmita, R. (2024). An investigation into science motivation, technology acceptance, and satisfaction intention during the transition to e-learning of prospective biology teachers. \u003cem\u003eCogent Education\u003c/em\u003e, \u003cem\u003e11\u003c/em\u003e(1), 2393065. https://doi.org/10.1080/2331186X.2024.2393065\u003c/li\u003e\n \u003cli\u003e*Heim, A. B. (2020). \u003cem\u003eUnderstanding\u0026nbsp;\u003c/em\u003e\u003cem\u003elearner-centeredness and student engagement in undergraduate biology education\u003c/em\u003e (Doctoral dissertation, University of Northern Colorado).\u003c/li\u003e\n \u003cli\u003eHenson, R. K., \u0026amp; Thompson, B. (2002). Characterizing measurement error in scores across studies: Some recommendations for conducting \u0026ldquo;reliability generalization\u0026rdquo; studies. \u003cem\u003eMeasurement and Evaluation in Counseling and Development\u003c/em\u003e, \u003cem\u003e35\u003c/em\u003e(2), 113\u0026ndash;127. https://doi.org/10.1080/07481756.2002.12069054\u003c/li\u003e\n \u003cli\u003e*Hibbard, L., Sung, S., \u0026amp; Wells, B. (2016). Examining the effectiveness of a semi-self-paced flipped learning format in a college general chemistry sequence. \u003cem\u003eJournal of Chemical Education\u003c/em\u003e, \u003cem\u003e93\u003c/em\u003e(1), 24\u0026ndash;30. https://doi.org/10.1021/acs.jchemed.5b00592\u003c/li\u003e\n \u003cli\u003eHiggins, J. P., Thompson, S. G., Deeks, J. J., \u0026amp; Altman, D. G. (2003). Measuring inconsistency in meta-analyses. \u003cem\u003eBmj\u003c/em\u003e, \u003cem\u003e327\u003c/em\u003e(7414), 557\u0026ndash;560.\u003c/li\u003e\n \u003cli\u003eHowell, R. T., \u0026amp; Shields, A. L. (2008). The file drawer problem in reliability generalization: A strategy to compute a fail-safe N with reliability coefficients. \u003cem\u003eEducational and Psychological Measurement\u003c/em\u003e, \u003cem\u003e68\u003c/em\u003e(1), 120\u0026ndash;128.\u003c/li\u003e\n \u003cli\u003e*Huda, N., \u0026amp; Rohaeti, E. (2023). The factors that influence the motivation to learn chemistry of upper-secondary school students in Indonesia. \u003cem\u003eJournal of Baltic Science Education\u003c/em\u003e, \u003cem\u003e22\u003c/em\u003e(4), 615\u0026ndash;630. http://dx.doi.org/10.33225/jbse/23.22.615\u003c/li\u003e\n \u003cli\u003e*Hugerat, M., Kortam, N., Maroun, N. T., \u0026amp; Basheer, A. (2020). The educational effectiveness of didactical games in project-based science learning among 5th grade students. \u003cem\u003eEurasia Journal of Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e16\u003c/em\u003e(10), em1888. https://doi.org/10.29333/ejmste/8490\u003c/li\u003e\n \u003cli\u003e*Iyer, R. D. (2017). Uncovering students\u0026apos; motivation for learning science: Gender differences in Mumbai. \u003cem\u003eGlobal Business and Management Research\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(3), 16.\u003c/li\u003e\n \u003cli\u003e*Kahraman, S. (2023). The use of dynamic computer visualizations integrated with the poe sequence: its effect on learners\u0026rsquo; understanding, retention, and motivation. \u003cem\u003eCanadian Journal of Science, Mathematics and Technology Education\u003c/em\u003e, \u003cem\u003e23\u003c/em\u003e(2), 179\u0026ndash;209. https://doi.org/10.1007/s42330-023-00284-z\u003c/li\u003e\n \u003cli\u003e*Kulesza, A. E., Imtiaz, S., \u0026amp; Bernot, K. M. (2022). Building connections to biology and community through service-learning and research experiences. \u003cem\u003eJournal of Microbiology \u0026amp; Biology Education\u003c/em\u003e, \u003cem\u003e23\u003c/em\u003e(3), e00082\u0026ndash;22. https://doi.org/10.1128/jmbe.00082-22\u003c/li\u003e\n \u003cli\u003e*Kwarikunda, D., Schiefele, U., Ssenyonga, J., \u0026amp; Muwonge, C. M. (2020). The relationship between motivation for, and interest in, learning physics among lower secondary school students in Uganda. \u003cem\u003eAfrican Journal of Research in Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e24\u003c/em\u003e(3), 435\u0026ndash;446. https://doi.org/10.1080/18117295.2020.1841961\u003c/li\u003e\n \u003cli\u003e*Kwarikunda, D., Schiefele, U., Ssenyonga, J., \u0026amp; Muwonge, C. M. (2021). Secondary school students\u0026rsquo; motivation profiles for physics learning: Relations with cognitive learning strategies, gender, attitudes and individual interest. \u003cem\u003eAfrican Journal of Research in Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e25\u003c/em\u003e(2), 197\u0026ndash;210. https://doi.org/10.1080/18117295.2021.1956720\u003c/li\u003e\n \u003cli\u003e*Lawson, D. (2020). \u003cem\u003eMotivating critical thinkers in fourth grade science: action research promoting 21st century skills through technology integrated project-based learning\u0026nbsp;\u003c/em\u003e(Doctoral dissertation, University of South Carolina).\u003c/li\u003e\n \u003cli\u003e*Lee, G. G., \u0026amp; Mun, S. (2023). From science motivation to science identity: The mediating effect of science achievement according to gender. \u003cem\u003eEurasia Journal of Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e19\u003c/em\u003e(10), em2341. https://doi.org/10.29333/ejmste/13633\u003c/li\u003e\n \u003cli\u003e*Lei, Y. (2024). Unlocking the secrets of STEM success: Exploring the interplay of motivation to learn science, self-regulation, and emotional intelligence from a perspective of self-determination theory. \u003cem\u003eLearning and Motivation\u003c/em\u003e, \u003cem\u003e87\u003c/em\u003e, 102012. https://doi.org/10.1016/j.lmot.2024.102012\u003c/li\u003e\n \u003cli\u003eL\u0026oacute;pez-Ib\u0026aacute;\u0026ntilde;ez, C., L\u0026oacute;pez-Nicol\u0026aacute;s, R., Bl\u0026aacute;zquez-Rinc\u0026oacute;n, D. M., \u0026amp; S\u0026aacute;nchez-Meca, J. (2024). Reliability generalization meta-analysis: Comparing different statistical methods. \u003cem\u003eCurrent Psychology\u003c/em\u003e, \u003cem\u003e43\u003c/em\u003e(20), 18275\u0026ndash;18293. https://doi.org/10.1007/s12144-023-05604-y\u003c/li\u003e\n \u003cli\u003eMeulenbroeks, R., van Rijn, R., \u0026amp; Reijerkerk, M. (2024). Fostering secondary school science students\u0026rsquo; intrinsic motivation by inquiry-based learning. \u003cem\u003eResearch in Science Education\u003c/em\u003e, \u003cem\u003e54\u003c/em\u003e(3), 339\u0026ndash;358. https://doi.org/10.1007/s11165-023-10139-0\u003c/li\u003e\n \u003cli\u003e*Migalang, G. M. M., \u0026amp; Azuelo, A. G. (2020). Students\u0026rsquo; cognitive learning and motivation through hybrid instructional strategy. \u003cem\u003eJournal of Science \u0026amp; Mathematics Education in Southeast Asia\u003c/em\u003e, \u003cem\u003e43\u003c/em\u003e.\u003c/li\u003e\n \u003cli\u003e*Moro, K. C., \u0026amp; Billote, W. J. S. M. (2023). Integrating Ivatan indigenous games to learning module in physics: its effect to student understanding, motivation, attitude, and scientific sublime. \u003cem\u003eScience Education International\u003c/em\u003e, \u003cem\u003e34\u003c/em\u003e(1), 3\u0026ndash;14. https://doi.org/10.33828/sei.v34.i1.1\u003c/li\u003e\n \u003cli\u003eNg, D. T. K., \u0026amp; Chu, S. K. W. (2021). Motivating students to learn STEM via engaging flight simulation activities. \u003cem\u003eJournal of Science Education and Technology\u003c/em\u003e, \u003cem\u003e30\u003c/em\u003e(5), 608\u0026ndash;629. https://doi.org/10.1007/s10956-021-09907-2\u003c/li\u003e\n \u003cli\u003eOrganization for Economic Cooperation and Development (OECD). (2007). PISA 2006: Science Competencies for Tomorrow\u0026rsquo;s World (Volume I: Analysis). Paris.\u003c/li\u003e\n \u003cli\u003ePark, S., \u0026amp; Kim, S. (2021). Is sustainable online learning possible with gamification? The effect of gamified online learning on student learning. \u003cem\u003eSustainability\u003c/em\u003e, \u003cem\u003e13\u003c/em\u003e(8), 4267. https://doi.org/10.3390/su13084267\u003c/li\u003e\n \u003cli\u003e*Pope, A. S. (2018). \u003cem\u003eScience attrition among high-performing college students: An analysis of motivational factors to improve retention strategies\u003c/em\u003e (Doctoral dissertation, Tennessee State University).\u003c/li\u003e\n \u003cli\u003e*Rachmatullah, A., \u0026amp; Ha, M. (2019). Indonesian and Korean high school students\u0026rsquo; disparities in science learning orientations: an approach to multi-group structural equation modeling. \u003cem\u003eAsia-Pacific Science Education\u003c/em\u003e, \u003cem\u003e5\u003c/em\u003e(1), 1\u0026ndash;17. https://doi.org/10.1186/s41029-019-0048-5\u003c/li\u003e\n \u003cli\u003e*Rachmatullah, A., Roshayanti, F., Shin, S., Lee, J. K., \u0026amp; Ha, M. (2018). The secondary-student science learning motivation in Korea and Indonesia. \u003cem\u003eEURASIA Journal of Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e14\u003c/em\u003e(7), 3123\u0026ndash;3141. https://doi.org/10.29333/ejmste/91665\u003c/li\u003e\n \u003cli\u003eRahmayanti, F., Irmagita, A., Wardhany, I. I., \u0026amp; Gunawan, B. (2020). Measurement of oral medicine learning motivation in dental students using the Indonesian version of Science Motivation Questionnaire II (SMQ-II). \u003cem\u003ePesquisa Brasileira em Odontopediatria e Cl\u0026iacute;nica Integrada\u003c/em\u003e, \u003cem\u003e20\u003c/em\u003e, e5486. https://doi.org/10.1590/pboci.2020.136\u003c/li\u003e\n \u003cli\u003e*Razali, F. (2021). Exploring crucial factors of an interest in STEM career model among secondary school students. \u003cem\u003eInternational Journal of Instruction, 14\u003c/em\u003e(2), 385\u0026ndash;404. https://doi.org/10.29333/iji.2021.14222a\u003c/li\u003e\n \u003cli\u003e*Razali, F., Manaf, U. K. A., \u0026amp; Ayub, A. F. M. (2020). STEM education in Malaysia towards developing a human capital through motivating science subject. \u003cem\u003eInternational Journal of Learning, Teaching and Educational Research\u003c/em\u003e, \u003cem\u003e19\u003c/em\u003e(5), 411\u0026ndash;422. https://doi.org/10.26803/ijlter.19.5.25\u003c/li\u003e\n \u003cli\u003eR Core Team (2025). \u003cem\u003eA language and environment for statistical computing\u003c/em\u003e. R foundation for statistical computing, Vienna, Austria. https://www.R-project.org\u003c/li\u003e\n \u003cli\u003e*Reece, A. J., \u0026amp; Butler, M. B. (2017). Virtually the same: A comparison of STEM students\u0026rsquo; content knowledge, course performance, and motivation to learn in virtual and face-to-face introductory biology laboratories. \u003cem\u003eJournal of College Science Teaching\u003c/em\u003e, \u003cem\u003e46\u003c/em\u003e(3), 83. https://doi.org/10.2505/4/jcst17_046_03_83\u003c/li\u003e\n \u003cli\u003e*Riccitelli, M. (2015). \u003cem\u003eScience identity\u0026apos;s influence on Community College Students\u0026apos; engagement, persistence, and performance in biology\u003c/em\u003e (Doctoral dissertation, Northcentral University).\u003c/li\u003e\n \u003cli\u003eRodriguez, M. C., \u0026amp; Maeda, Y. (2006).\u003cem\u003e\u0026nbsp;\u003c/em\u003eMeta-analysis of coefficient alpha.\u003cem\u003e\u0026nbsp;Psychological Methods, 11,\u0026nbsp;\u003c/em\u003e306\u0026ndash;322\u003cem\u003e.\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003eRoberts, D. (2007). \u003cem\u003eScientific literacy/science literacy.\u003c/em\u003e In S. K. Abell \u0026amp; N. G. Lederman (Eds.), Handbook of Research on Science Education (pp. 729\u0026ndash;780). Mahwah, NJ: Lawrence.\u003c/li\u003e\n \u003cli\u003e*Salta, K., \u0026amp; Koulougliotis, D. (2015). Assessing motivation to learn chemistry: adaptation and validation of Science Motivation Questionnaire II with Greek secondary school students. \u003cem\u003eChemistry Education Research and Practice\u003c/em\u003e, \u003cem\u003e16\u003c/em\u003e(2), 237\u0026ndash;250. https://doi.org/10.1039/C4RP00196F\u003c/li\u003e\n \u003cli\u003e*Salta, K., \u0026amp; Koulougliotis, D. (2020). Domain specificity of motivation: chemistry and physics learning among undergraduate students of three academic majors. \u003cem\u003eInternational Journal of Science Education\u003c/em\u003e, \u003cem\u003e42\u003c/em\u003e(2), 253\u0026ndash;270. https://doi.org/10.1080/09500693.2019.1708511\u003c/li\u003e\n \u003cli\u003eS\u0026aacute;nchez-Meca, J., L\u0026oacute;pez-L\u0026oacute;pez, J. A. \u0026amp; L\u0026oacute;pez-Pina, J. A. (2013). Some recommended statistical analytic practices when reliability generalization studies are conducted. \u003cem\u003eBritish Journal of Mathematical and Statistical Psychology, 66\u003c/em\u003e(3), 402\u0026ndash;425. 10.1111/j.2044-8317.2012.02057. https://doi.org/10.1111/j.2044-8317.2012.02057.x\u003c/li\u003e\n \u003cli\u003eSanchez-Meca, J., Rubio-Aparicio, M., L\u0026oacute;pez-Pina, J., Nu\u0026ntilde;ez-Nu\u0026ntilde;ez, R.M., \u0026amp; Marın-Martınez, F. (2015).\u003cem\u003e\u0026nbsp;\u003c/em\u003eThe phenomenon of reliability induction in the social and health sciences.\u003cem\u003e\u0026nbsp;XIV Congress of Methodology of the Social and Health Sciences.\u003c/em\u003e\u003c/li\u003e\n \u003cli\u003eS\u0026aacute;nchez‐Meca, J., Mar\u0026iacute;n‐Mart\u0026iacute;nez, F., L\u0026oacute;pez‐L\u0026oacute;pez, J. A., N\u0026uacute;\u0026ntilde;ez‐N\u0026uacute;\u0026ntilde;ez, R. M., Rubio‐Aparicio, M., L\u0026oacute;pez‐Garc\u0026iacute;a, J. J., ... \u0026amp; L\u0026oacute;pez‐Nicol\u0026aacute;s, R. (2021). Improving the reporting quality of reliability generalization meta‐analyses: The REGEMA checklist. \u003cem\u003eResearch Synthesis Methods\u003c/em\u003e, \u003cem\u003e12\u003c/em\u003e(4), 516\u0026ndash;536. https://doi.org/10.1002/jrsm.1487\u003c/li\u003e\n \u003cli\u003eSandoval‐Lentisco, A., L\u0026oacute;pez‐Nicol\u0026aacute;s, R., L\u0026oacute;pez‐L\u0026oacute;pez, J. A., \u0026amp; S\u0026aacute;nchez‐Meca, J. (2023). Florida Obsessive‐Compulsive Inventory and Children\u0026apos;s Florida Obsessive Compulsive Inventory: A reliability generalization meta‐analysis. \u003cem\u003eJournal of Clinical Psychology\u003c/em\u003e, \u003cem\u003e79\u003c/em\u003e(1), 28\u0026ndash;42. https://doi.org/10.1002/jclp.23416\u003c/li\u003e\n \u003cli\u003eScherer, R., \u0026amp; Teo, T. (2020). A tutorial on the meta-analytic structural equation modeling of reliability coefficients. \u003cem\u003ePsychological methods\u003c/em\u003e, \u003cem\u003e25\u003c/em\u003e(6), 747.\u003c/li\u003e\n \u003cli\u003e*Schumm, M. F., \u0026amp; Bogner, F. X. (2016). Measuring adolescent science motivation. \u003cem\u003eInternational Journal of Science Education\u003c/em\u003e, \u003cem\u003e38\u003c/em\u003e(3), 434\u0026ndash;449. https://doi.org/10.1080/09500693.2016.1147659\u003c/li\u003e\n \u003cli\u003e*Schmid, S., \u0026amp; Bogner, F. X. (2017). How an inquiry-based classroom lesson intervenes in science efficacy, career-orientation and self-determination. \u003cem\u003eInternational Journal of Science Education\u003c/em\u003e, \u003cem\u003e39\u003c/em\u003e(17), 2342\u0026ndash;2360. https://doi.org/10.1080/09500693.2017.1380332\u003c/li\u003e\n \u003cli\u003e*Sch\u0026ouml;nfelder, M. L., \u0026amp; Bogner, F. X. (2020). Between science education and environmental education: How science motivation relates to environmental values. \u003cem\u003eSustainability\u003c/em\u003e, \u003cem\u003e12\u003c/em\u003e(5), 1968. https://doi.org/10.3390/su12051968\u003c/li\u003e\n \u003cli\u003e*Shin, S., Rachmatullah, A., Ha, M., \u0026amp; Lee, J. K. (2018). A longitudinal trajectory of science learning motivation in Korean high school students. \u003cem\u003eJournal of Baltic Science Education\u003c/em\u003e, \u003cem\u003e17\u003c/em\u003e(4), 674\u0026ndash;687. https://doi.org/10.33225/jbse/18.17.674\u003c/li\u003e\n \u003cli\u003eShin, D. D., Lee, M., Jung, S. J., \u0026amp; Bong, M. (2023). Relative effects of classroom utility value intervention on the science motivation of girls and boys. \u003cem\u003eResearch in Science Education\u003c/em\u003e, \u003cem\u003e53\u003c/em\u003e(3), 593\u0026ndash;612. https://doi.org/10.1007/s11165-022-10070-w\u003c/li\u003e\n \u003cli\u003eThompson, B. (2002). \u003cem\u003eScore reliability: Contemporary thinking on reliability issues\u003c/em\u003e. Sage.\u003c/li\u003e\n \u003cli\u003eThompson, B., \u0026amp; Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. \u003cem\u003eEducational and Psychological Measurement, 60\u003c/em\u003e(2), 174\u0026ndash;195. https://doi.org/10.1177/0013164400602\u003c/li\u003e\n \u003cli\u003e*Toma, R. B., Cordeiro Bizerra, A. M., Y\u0026aacute;nez, I., \u0026amp; Meneses Villagr\u0026aacute;, J. \u0026Aacute;. (2023). Cross-cultural adaptation of the Science Motivation Questionnaire II (SMQ-II) for Portuguese-speaking Brazilian secondary school students. \u003cem\u003eRevista Latinoamericana de Psicolog\u0026iacute;a\u003c/em\u003e, \u003cem\u003e55\u003c/em\u003e, 109\u0026ndash;119. https://doi.org/10.14349/rlp.2023.v55.13\u003c/li\u003e\n \u003cli\u003eTosun, C. (2013). Kimya Motivasyon Anketi-II\u0026rsquo;nin T\u0026uuml;rk\u0026ccedil;eye uyarlanması. \u003cem\u003eErzincan \u0026Uuml;niversitesi Eğitim Fak\u0026uuml;ltesi Dergisi\u003c/em\u003e, \u003cem\u003e15\u003c/em\u003e(1), 173\u0026ndash;202.\u003c/li\u003e\n \u003cli\u003eVacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. \u003cem\u003eEducational and Psychological Measurement\u003c/em\u003e, \u003cem\u003e58\u003c/em\u003e(1), 6\u0026ndash;20.\u003c/li\u003e\n \u003cli\u003eVacha-Haase, T., Kogan, L. R., \u0026amp; Thompson, B. (2000). Sample compositions and variabilities in published studies versus those in test manuals: Validity of score reliability inductions. \u003cem\u003eEducational and Psychological Measurement\u003c/em\u003e, \u003cem\u003e60\u003c/em\u003e(4), 509\u0026ndash;522. https://doi.org/10.1177/00131640021970682\u003c/li\u003e\n \u003cli\u003eVacha-Haase, T., \u0026amp; Thompson, B. (2011). Score reliability: A retrospective look back at 12 years of reliability generalization studies. \u003cem\u003eMeasurement and Evaluation in\u003c/em\u003e \u003cem\u003eCounseling and Development, 44\u003c/em\u003e, 159\u0026ndash;168. http://doi.org/10.1177/0748175611409845\u003c/li\u003e\n \u003cli\u003eVan Vo, D., \u0026amp; Csap\u0026oacute;, B. (2022). Exploring students\u0026rsquo; science motivation across grade levels and the role of inductive reasoning in science motivation. \u003cem\u003eEuropean Journal of Psychology of Education\u003c/em\u003e, \u003cem\u003e37\u003c/em\u003e(3), 807\u0026ndash;829. http://dx.doi.org/10.1007/s10212-021-00568-8\u003c/li\u003e\n \u003cli\u003eViechtbauer, W. (2010). Conducting meta-analyses in R with the \u003cem\u003emetafor\u003c/em\u003e package. \u003cem\u003eJournal of Statistical Software\u003c/em\u003e, 36, 1\u0026ndash;48.\u003c/li\u003e\n \u003cli\u003e*Watt, K. (2020). \u003cem\u003eGuided simulation: A new model of game-based pedagogy for non-stem students in the community college\u003c/em\u003e (Doctoral dissertation, Gwynedd Mercy University).\u003c/li\u003e\n \u003cli\u003eWilkinson, L. (1999). Statistical methods in psychology journals: Guidelines and explanations. \u003cem\u003eAmerican Psychologist\u003c/em\u003e, \u003cem\u003e54\u003c/em\u003e(8), 594.\u003c/li\u003e\n \u003cli\u003e*Willoughby, B. R., Flack, N. A., Bird, R. J., \u0026amp; Woodley, S. J. (2024). Motivation to learn in university science students studying anatomy: A mixed‐methods analysis of what drives learning. \u003cem\u003eAnatomical sciences education\u003c/em\u003e, \u003cem\u003e17\u003c/em\u003e(4), 818\u0026ndash;830. https://doi.org/10.1002/ase.2416\u003c/li\u003e\n \u003cli\u003eYang, Y., \u0026amp; Green, S. B. (2011).\u003cem\u003e\u0026nbsp;\u003c/em\u003eCoefficient alpha: A reliability coefficient for the 21st century?\u003cem\u003e\u0026nbsp;Journal of Psychoeducational Assessment, 29\u003c/em\u003e(4),\u003cem\u003e\u0026nbsp;\u003c/em\u003e377\u0026ndash;392. https://doi.org/10.1177/0734282911406668\u003c/li\u003e\n \u003cli\u003eYin, P., \u0026amp; Fan, X. (2000). Assessing the reliability of Beck Depression Inventory scores: Reliability generalization across studies. \u003cem\u003eEducational and psychological measurement\u003c/em\u003e, \u003cem\u003e60\u003c/em\u003e(2), 201\u0026ndash;223.\u003c/li\u003e\n \u003cli\u003e*You, H. S., Kim, K., Black, K., \u0026amp; Min, K. W. (2018). Assessing science motivation for college students: Validation of the Science Motivation Questionnaire II using the rasch-andrich rating scale model. \u003cem\u003eEurasia Journal of Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e14\u003c/em\u003e(4), 1161\u0026ndash;1173. https://doi.org/10.29333/ejmste/81821\u003c/li\u003e\n \u003cli\u003e*Zhdanov, S. P., Ishmuradova, A. M., Zakharova, V. L., Belous, S. V., \u0026amp; Grishnova, E. E. (2022). Validation and adaptation of the questionnaire on science motivation in the Russian context. \u003cem\u003eEurasia Journal of Mathematics, Science and Technology Education\u003c/em\u003e, \u003cem\u003e18\u003c/em\u003e(8). https://doi.org/10.29333/ejmste/12219 Note: References with an asterisk (*) indicate studies included in the reliability generalization meta-analysis in this study.\u003c/li\u003e\n\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTables 1 to 4 are available in the Supplementary Files section\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Leuphana University of Lüneburg","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"Reliability generalization, Science Motivation, Science Education, Meta-analysis","lastPublishedDoi":"10.21203/rs.3.rs-6419938/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6419938/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eScience motivation is among the most important constructs affecting students\u0026rsquo; science learning and scientific thinking skills. One commonly utilized scale to measure students\u0026rsquo; science motivation is the Science Motivation Questionnaire II (SMQ-II), a 25-item, Likert-type, self-report scale. This study aimed to conduct a reliability generalization meta-analysis of the scale considering the REGEMA guideline. Studies included in the analysis implemented the instrument between 2011\u0026ndash;2024 and reported a Cronbach alpha value. Reliability evidence from 49 studies reporting a coefficient for the total score or at least for one of the five subscale scores was analyzed using a random effects model and Bonnet\u0026rsquo;s transformation. The pooled Cronbach alpha reliability coefficient was 0.94 for the total score and ranged from 0.83\u0026ndash;0.88 for the subscale scores. Moderator analyses showed generally similar reliability estimates across studies, with different study types, languages, sample types, or sample sizes for the subscales. However, the test version, sample type, sample size, and female representation in the sample showed differences in reliability estimates for the total score. The empirical evidence from the SMQ-II\u0026rsquo;s first 13 years reports high internal consistency across the scale\u0026rsquo;s scores.\u003c/p\u003e","manuscriptTitle":"A Reliability Generalization Meta-Analysis of the Science Motivation Questionnaire II","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-04-11 08:08:48","doi":"10.21203/rs.3.rs-6419938/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"6e02312d-448a-4e73-ad56-681ba1c04b5e","owner":[],"postedDate":"April 11th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":46973737,"name":"Educational Psychology"}],"tags":[],"updatedAt":"2025-04-11T08:08:48+00:00","versionOfRecord":[],"versionCreatedAt":"2025-04-11 08:08:48","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6419938","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6419938","identity":"rs-6419938","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}