Students’ Online Credibility Evaluation Skills Across Four Grades Representing Three Educational Levels: Factorial Invariance and Cross-Sectional Comparisons

Students’ Online Credibility Evaluation Skills Across Four Grades Representing Three Educational Levels: Factorial Invariance and Cross-Sectional Comparisons | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Research Article Students’ Online Credibility Evaluation Skills Across Four Grades Representing Three Educational Levels: Factorial Invariance and Cross-Sectional Comparisons Laura Kanniainen, Asko Tolvanen, Eija Räikkönen, Riikka Anttonen, and 2 more This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-7815537/v1 This work is licensed under a CC BY 4.0 License Status: Posted Version 1 posted You are reading this latest preprint version Abstract We sought to validate a web-based online credibility evaluation task across four grade levels and three educational stages. The aim is to provide a robust tool for assessing students’ online credibility evaluation skills and grade-level differences in skills while controlling for text reading order, prior topic knowledge, and reading fluency. The sample comprised 728 students from primary (fourth and sixth graders), lower secondary (eighth graders), and upper secondary (10th graders) education in Finland. Students evaluated four online texts representing different genres: a popular science text, a science newspaper article, a layperson’s blog text, and a commercial text. Each text was evaluated from three perspectives: author expertise, author benevolence, and quality of evidence. The results confirmed the factorial invariance of the task across grade levels, supporting the validity of comparisons between the grade levels. Older students performed higher than younger ones in evaluating less credible online texts (i.e., the blog and commercial text). By offering a theoretically grounded and methodologically robust evaluation approach, our findings contribute to advancing assessment practices in the digital age. Educational Psychology credibility evaluation online evaluation sourcing digital literacies factorial invariance Figures Figure 1 Introduction In today’s digital landscape, children and adolescents are exposed to a wide range of online content, including articles, news, blogs, social media posts, and advertisements (Smahel et al., 2020). While online content provides opportunities for learning and connecting, it also poses a significant risk of encountering misinformation (Ecker et al., 2022). Given the prevalence of inaccurate online information, exemplified by the recent “infodemic” (Rocha et al., 2021), students must be equipped with sufficient skills to assess the quality of online information, which we call online credibility evaluation. To tailor instruction to appropriately support the development of students’ online credibility evaluation skills, it is necessary to understand the level of students’ skills at different stages of their schooling. Although there is a substantial body of research on students’ online credibility evaluation skills, it is limited in two important respects. First, previous research has employed various measures to assess students’ credibility evaluation skills, such as printed web-alike materials (e.g., Macedo-Rouet et al., 2019; Pérez et al., 2018), researcher-designed online texts (Kiili et al., 2023), and authentic online texts (Barzilai & Zohar, 2012; Coiro et al., 2015; Hämäläinen et al., 2021). In addition, students have indicated their credibility evaluations with different items, such as multiple choice, rating scales, ranking and open items, or a combination. These differences in measurement instruments make it challenging to compare findings on students’ credibility evaluation skills across studies and interpret them across different grade levels. Second, studies cross-sectionally examining credibility evaluation skills across the different grade levels have been conducted without considering the factorial invariance of measures used to assess those skills, leaving it unclear whether the measures used assessed the same construct across different grade levels (Macedo-Roeut et al., 2013; 2019; McGrew et al., 2018; Pieschl & Sivyer, 2021; Potocki et al., 2020; Salmerón et al., 2016). To address these issues, this study has two main aims. The first aim is to examine whether the previously identified structure of students’ online credibility evaluation skills (Kiili et al., 2023) is consistent across four grade levels, from primary to upper secondary education, and to validate an online credibility evaluation assessment task by testing its factorial invariance across these grade levels. The second aim is to investigate how students’ online credibility evaluation skills vary across the four grades representing three education levels. Thus, the validation of this newly developed measure will not only offer a methodological contribution but also shed light on students’ online credibility evaluation skills across grade levels, from primary to upper secondary education, offering valuable knowledge for designing effective instruction. Theoretical Framing of Online Credibility Evaluation We draw on several theoretical frameworks that are central to our exploration of students’ online credibility evaluation skills. Since readers often encounter multiple online texts of varying quality, the theory of online research and comprehension (Leu et al., 2015; 2019) underscores the evaluation of information as an essential component of reading online. Evaluation, which can concern relevance and credibility, may occur in all phases of online research and comprehension (see Kiili et al., 2021, for sourcing). While this theory situates evaluation as a vital part of reading online, other models or frameworks describe how readers evaluate credibility when confronted with conflicting information offline or online. According to the content–source integration model (Stadtler & Bromme, 2014), readers can use first- and second-hand evaluation to resolve conflicts when encountering contradictory scientific claims in multiple texts. Readers can employ first-hand evaluation to judge the veracity or accuracy of knowledge claims (i.e., text content) by relying on prior knowledge. They can also employ second-hand evaluation to judge the trustworthiness of information sources, such as the text’s author or publisher. When readers encounter familiar topics, they tend to rely on first-hand evaluation, but with less familiar topics, they tend to emphasize second-hand evaluation (e.g., Bråten et al., 2018). The bidirectional model of first- and second-hand evaluation strategies (Barzilai et al., 2020) extends the content–source integration model by demonstrating that readers employ multiple strategies that can impact content and source evaluation in a bidirectional manner. This model differentiates three first-hand evaluation strategies. First, knowledge-based strategies refer to the evaluation of knowledge claims against one’s existing knowledge or beliefs. Second, discourse-based strategies involve evaluating the grounding of claims, including the logic of reasoning and the quality of evidence. For instance, research has shown that even by the time children reach second grade, they begin to understand the flawed nature of circular reasoning in arguments and explanations (Baum et al., 2008). Third, corroboration refers to validating claims with information available from other credible resources. Second-hand evaluation strategies focus on evaluating author expertise and benevolence (Barzilai et al., 2020; Stadtler & Bromme, 2014). Author expertise reflects competence in sharing accurate information, while benevolence refers to authors acting in readers’ interests without pursuing personal gains (Pérez et al., 2018; Thomm & Bromme, 2016). Readers may question author benevolence if, for example, they identify author commercial interests or one-sided argumentation aligned with the author’s personal aims (Kiili et al., 2023). In the present study, models depicting first- and second-hand evaluation informed our task design. We incorporate conflicting texts into our online credibility evaluation task and assess students’ abilities to evaluate author expertise, author benevolence, and the quality of evidence of these texts. Building on the concepts of first- and second-hand evaluation, the critical online resource evaluation framework (Forzani, 2020; Forzani et al., 2022) focuses on credibility evaluation on the internet and introduces an additional layer, tertiary evaluation. In tertiary evaluation, readers can consider how the context (i.e., the temporal, social, and political settings of the text) in which the content is presented reflects the text’s credibility. One available cue for tertiary evaluation is the text genre (Flanagin & Metzger, 2007; List et al., 2017). Readers can use their online genre knowledge, such as knowledge of a specific genre’s organizational structures, conventions, and publication practices, to evaluate the context’s trustworthiness (Corrigan & Slomp, 2021). This additional layer is considered in our text design because the designed texts represented various online genres. Finally, previous research has shown that readers’ ability to evaluate online information may be a multidimensional construct (Kanniainen et al., 2022; Kiili et al., 2023; Kulju et al., 2024). For instance, Kiili et al. (2018, 2023) found that students needed different online evaluation skills to confirm the more credible online texts and to question the less credible ones. This was also evident with a person-centered approach, in which a latent profile analysis revealed a group of adolescent online readers with particular difficulty in questioning the credibility of online information (Kanniainen et al., 2022). Moreover, Kiili et al. (2023) demonstrated that both text genre and text credibility played a role in students’ online credibility evaluation. In the present study, we are interested in how students representing different educational levels perform in evaluating more and less credible online texts representing various genres. Developmental Aspects of Online Credibility Evaluation Skills The development of students’ online credibility evaluation skills can be approached using Alexander’s (2005) developmental model of reading , which describes how individuals’ reading skills develop throughout their lifespan. As readers mature, their strategic processing shifts from superficial to deeper levels of processing. In this developmental path, readers’ topic and domain knowledge becomes crucial for building competence and expertise in reading. Along with gaining new knowledge during schooling, exposure to diverse texts and perspectives further refines students’ strategies for evaluating text content and information sources, both of which are essential for successfully evaluating online credibility (Svedholm-Häkkinen et al., 2025). Regarding online credibility evaluation, there is a lack of understanding of how students’ skills develop during schooling. Namely, most research on students’ online credibility evaluation skills has focused on primary school (e.g., Barzilai & Zohar, 2012; Kanniainen et al., 2019), secondary school (e.g., Coiro et al., 2015; van der Eem et al., 2024), or university (e.g., Barzilai et al., 2015; Fendt et al., 2023) students. In fact, few studies have compared different grade levels, even cross-sectionally, and those that did derived their findings from relatively small sample sizes ( n < 300). For example, Macedo-Rouet et al. (2013) included fourth and fifth graders in their study (Experiment 1: N = 98, Experiment 2: N = 96; ages 9–11 years). Although they found that fifth graders performed higher than fourth graders in identifying the most knowledgeable source of the texts (Experiment 1), they compared only two grade levels. Salmerón et al. (2016) examined students from fifth grade to university level (Experiment 1: N = 137, Experiment 2: N = 277; ages 10–21 years). University students more frequently cited source attributes and relied less frequently on personal opinions to justify their forum post selections than younger students, especially when forum posts contained contradictory information (cf. Experiment 1 vs. Experiment 2). Similarly, Macedo-Rouet et al. (2019) studied sixth to 10th graders (Experiment 1: N = 57, Experiment: 2: N = 36; ages 12–16 years) and found that younger students identified expert sources less spontaneously and used source cues more superficially than older students. Pieschl and Sivyer (2021) found differences between seventh, ninth, and 11th graders ( N = 218; ages 12–17 years). Ninth and 11th graders were able to distinguish credible blog posts from noncredible ones, while seventh graders still struggled. Potocki et al. (2020) identified similar trends in a study of fifth, seventh, and ninth graders and university students ( N = 245, ages 10–19 years), observing an increased ability to distinguish between proficient and less proficient authors and a shift toward source-based justifications from content-based ones. To the best of our knowledge, only one study with a large sample size has examined students’ online credibility evaluation skills. McGrew et al. (2018) explored credibility evaluations among students from middle school to college ( N = 894, no age range available) and found that students struggled to effectively evaluate the sources and evidence across different grade levels. However, the results were not directly comparable across grade levels because the tasks and implementation methods (paper or online) varied among students from different stages of education. It is noteworthy that none of the aforementioned studies tested the factorial invariance of the tasks used in their research designs. In other words, it remains unclear whether the designed tasks consistently measured students’ credibility evaluation skills across various educational levels and grades. Thus, it cannot be determined whether the findings reflected actual differences in skills since they could have been biased if the tasks failed to measure the same construct across age groups. Therefore, there is a need to validate the same credibility evaluation task across different grade levels to increase the validity of cross-sectional comparisons and to enable future longitudinal studies. Present Study In this study, we set out to validate an online credibility evaluation task for primary (fourth and sixth graders), lower secondary (eighth graders), and upper secondary (10th graders) school students. This task was previously validated for sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024). We aim to extend this validation and employ the task to compare students’ online credibility evaluation skills across the four grade levels. In the task, students read and evaluate four texts representing different genres: two genres typically representing more credible online texts (a popular science text and a science newspaper article) and two genres typically representing less credible ones (a layperson’s blog text and a commercial text). Students are asked to evaluate the texts’ credibility regarding three credibility aspects: author expertise, author benevolence, and quality of evidence. Kiili et al. (2023) found that these aspects were divided into factors based on the text genres, with both the first-order factor model (i.e., genre-based model) and the second-order factor model (i.e., the genre-based confirming and questioning model) fitting the data well. We use the former as our baseline model in this study because it provides a clear, stepwise approach to testing invariance. Our first goal is to answer Research Question 1: Does an online credibility evaluation task measure students’ abilities to evaluate online texts representing different genres similarly across the four grade levels? We expect that the students’ online credibility evaluations concerning the aforementioned aspects of credibility will reflect the four factors according to the text genres. We also anticipate that the genre-based factor structure will be similar across the four grade levels, reflecting findings from earlier studies conducted among sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024). Next, we seek to answer Research Question 2: How do students’ online credibility evaluation skills differ across the four grade levels after controlling for their prior topic knowledge and reading fluency? We expect that older students will perform higher than younger students in their online credibility evaluations across all online text genres. This assumption is based on theoretical considerations of reading development suggesting that older students can engage in deeper strategic processing than younger students (Alexander, 2005). Our assumptions are supported by previous empirical findings suggesting that older students perform higher than younger students, especially when they read contradictory information (Macedo-Rouet et al., 2019; McGrew et al., 2018; Pieschl & Sivyer, 2021). They are also supported by the learning objectives of the Finnish curriculums (Finnish National Agency for Education, 2016, 2019, 2024), which emphasize the teaching of multiliteracies across grade levels and subjects. As students mature, they are exposed to reading and evaluating offline and online texts with increasing levels of complexity, perspectives, and contexts; their online credibility evaluation skills should improve during schooling. In the analysis, we control for prior topic knowledge due to its theoretically well-established role in reading comprehension (Alexander, 2005; Kintsch, 1998), and previous research has shown that prior knowledge may contribute to online credibility evaluation (Forzani, 2018; Kammerer et al., 2021; Kiili et al., 2024). We also control for students’ reading fluency, since word-level reading skills can predict online credibility evaluation abilities in both primary and secondary school students (Hämäläinen et al., 2021; Macedo-Rouet et al., 2013, 2020). Methods Participants and Context A total of 728 students participated (50.6% girls, 48.6% boys, 0.8% non-binary), of whom 139 were fourth graders ( M = 10.50 years, SD = 0.31), 198 were sixth graders ( M = 12.57 years, SD = 0.38), 203 were eighth graders ( M = 14.64 years, SD = 0.33), and 188 were first-year general and vocational upper secondary school students ( M = 16.90 years, SD = 0.64; hereafter 10 th graders). Most of the students’ (91.5%) home language was Finnish, while 5.0% spoke Finnish and some other language at home. Only 3.1% spoke some language other than Finnish at home. All the students had a sufficiently good language level to study in mainstream education following the national curricula (Finnish National Agency for Education, 2016, 2019, 2024). Students were recruited from 42 classrooms representing seven comprehensive schools and from 12 classrooms whose teaching was organized by one consortium of general/vocational upper secondary education. After receiving an ethical statement from the Ethics Committee of the [BLINDED] Region, schools were recruited through phone or email contact with the school principals during the 2021–2022 school year. The principals forwarded the recruitment request to teachers. All the classes and students participated voluntarily. Students and their guardians, for those under 15 years, signed written consent forms for participation. In Finland, compulsory education begins with comprehensive school (grades 1–9), which includes primary (grades 1–6) and lower secondary levels (grades 7–9). After completing these levels, students continue to the upper secondary level, which usually lasts three years (corresponding to grades 10–12). They can choose between general education and vocational education and training. General education provides a broad academic foundation, leading to the Finnish matriculation examination and qualification for higher education. Vocational education, while offering access to higher education, equips students with essential skills in specific fields. Compulsory education continues until students turn 18 or complete their upper secondary education. Online Credibility Evaluation Task The online credibility evaluation task was created with the Critical Online Reading Research Environment (Kiili et al., 2023), where researchers can design web-based credibility evaluation tasks. Online Texts The task asked students to read and evaluate the credibility of four online texts about the health benefits of vitamins (see Table 1). Since the texts were researcher-designed, we manipulated the text content (accuracy of the main claim and quality of the evidence), the source trustworthiness (authors’ expertise and benevolence), and the text genre. Two of the texts can be considered more credible (a science newspaper article and a popular science text) and two less credible (a layperson’s blog text and a commercial text). The four texts were similar in length and were grouped into two text pairs, which considered the same subtopic with contradictory main claims. Subtopic 1 concerned vitamin C and the flu (vitamin C prevents / does not prevent the flu) and Subtopic 2 concerned the health benefits of multivitamins (multivitamins have health benefits / multivitamins have no health benefits if a person has a normal, healthy diet). To counterbalance the possible effects of the reading order, the students read the texts in two different orders, which were randomly assigned. Regarding Subtopic 1, Group 1 read the less credible text first and then the more credible text. Regarding Subtopic 2, Group 1 read the more credible text first and then the less credible text. Group 2 read the texts within the subtopics in reverse order. [TABLE 1] Task and Items During the task, the students were instructed by Max, an avatar fact-checker, to read and evaluate one text at a time. For each online text, students responded to three credibility evaluation items concerning (a) author expertise, (b) author benevolence, and (c) the quality of evidence supporting the text’s main claim. The students provided their credibility evaluations on a six-point rating scale (e.g., how much the author has expertise in the health effects of vitamins: 1 = hardly at all; 6 = very much), with their evaluations scored as follows. For the more credible texts, students received 2 points for using the top of the scale (5 or 6), 1 point for using the middle scale (3 or 4), and 0 points for using the bottom of the scale (1 or 2). For the less credible texts, the scoring was reversed. In addition to the credibility evaluation items, the task included identification items (identification of the author and the main claim and evidence) and justification items (justifications for the credibility evaluations). Although these multiple-choice items were not used in this study, the students responded to the identification items before the evaluation items. If a student did not answer the identification items correctly, Max provided the correct answer to ensure that the correct author or evidence had been evaluated. Prior Topic Knowledge Before students were instructed to undertake the credibility evaluation task, they responded to 10 true–false topic knowledge items about vitamins (e.g., Our body is able to produce vitamin C; Multivitamin pills are healthier than dietary vitamins; Spending time in the sun increases vitamin D ). One point was given for each correct response (max. 10 points). Items were embedded in the task environment and used as a control variable (a mean composite of a student’s prior topic knowledge) in the subsequent analyses. Since the prior topic knowledge items measured diverse aspects of vitamins, they violated the unidimensionality assumption required for both Cronbach’s alpha and McDonald’s omega (McNeish, 2018). Since knowledge tests often assess distinct components rather than a single construct (Taber, 2018), we do not report these reliability measures. The 10 items were validated by two medical experts, who checked and modified them if needed. Reading Fluency Measures We measured the students’ reading fluency using two standardized, time-limited paper-and-pencil tasks: a word-reading task, and a sentence-reading task (Lerkkanen et al., 2018). The word-reading task consisted of 80 items, each with four words and one picture. The students were instructed to connect the correct picture–word pair by drawing a line between a word and a picture and given two minutes to complete the task. The score was calculated as the number of correctly connected pairs minus 0.33 times the number of incorrect responses. The sentence-reading task consisted of 70 items, and in each item, the students read one sentence. The task was to conclude whether the content of the sentence was true or not. The sentences were designed in a way that made this distinction easily noticeable (e.g., true: a cow is an animal, false: a fish lives on land). Again, the students had two minutes to complete the task. The score was the number of correct answers minus the number of incorrect ones. A reading fluency factor (see the Analysis Strategy section) was formed based on these two tasks. McDonald’s reliability coefficient for the two reading fluency tasks was .89. The reading fluency factor was used as a control variable. Procedure Data were collected remotely via Microsoft Teams due to COVID-19 restrictions. For 10 th graders, data were collected during a 75-minute class, while younger students participated in two 45-minute sessions. Paper-and-pencil tasks were distributed to schools before the Teams sessions. At the beginning of each class, the researcher briefly introduced the structure of the tasks via Teams. First, the students completed two paper-and-pencil reading fluency tasks, with instructions provided through a one-minute video. The completed tasks were stored in a locked closet until retrieved by the researcher. Second, the students completed the credibility evaluation task, which was allotted 45 minutes, with an option to use their 15-minute recess. Students watched a 1-minute 49-seconds introductory video on how to log in and navigate the task environment, and then logged in using the given codes. The researcher monitored through student progress via the administrative version of the environment. Students completed the credibility evaluation task with computers/laptops at their own pace. During the task, the researcher’s camera and microphone were off, but communication with the teacher and classroom was maintained using Teams chat. For example, the teacher was informed when the first students finished or was instructed if some students were very slow in the task. After all students finished, the researcher thanked them and the teacher. Average task completion times were: 21:34 ( SD = 06:04) for fourth graders, 21:57 ( SD = 05:29) for sixth graders, 19:34 ( SD = 04:26) for eighth graders, and 14:49 ( SD = 04:38) for 10th graders. Notably, the fourth graders’ version of the task did not include justification items due to time constraints, resulting in a task completion time similar to that of older students. Within a month, the students and teachers received feedback. The students received encouraging and constructive feedback from Max, the avatar that guided them through the task. Teachers received class- and student-level feedback about their students’ performance on all tasks. Statistical Analyses Descriptive and reliability analyses as well as multivariate analysis of covariance (MANCOVA) were performed using SPSS Statistics 28. Multigroup confirmatory factor analyses were conducted using Mplus Version 8.9 (Muthén & Muthén, 1998–2017). Before the actual analyses, we noticed that of the 739 original participants, three had interrupted the credibility evaluation task for random reasons (e.g., a dental appointment). These participants were removed from the final dataset. Also, one adult student from the general/vocational upper secondary level was removed based on their age (37 years; cf. Participants and Context section). Seven other participants were removed based on the time spent on the task (±3 standard deviations from the average). Finally, as the students were nested within 54 different classrooms, intraclass correlation coefficients (ICCs) were calculated for the credibility evaluation items. The ICCs showed that 2.5%–11.2% of the variance in the items was explained by differences at the classroom level. We accounted for this by using class as a clustering variable with the COMPLEX option in further analyses in Mplus. Factorial Invariance of the Online Credibility Evaluation Task Across Different Grade Levels Research Question 1 explores the measurement invariance of the factor structure of the online credibility evaluation task, representing different genres, across fourth, sixth, eighth, and 10th graders. Multigroup confirmatory factor analyses with the weighted least square mean and variance adjusted estimator for categorical credibility evaluation variables were used to test factorial invariance. The factorial invariance tests were conducted in four phases: (a) configural invariance, (b) metric invariance, (c) scalar invariance, and (d) strict invariance (Putnick & Bornstein, 2016). In each phase, the factor model of the previous phase served as a reference. In the first phase, we assessed configural invariance (Model 1) by freely estimating factor loadings and thresholds of the credibility evaluation items while fixing the residual variances to 1. Further, the latent variances were constrained to be 1 and the latent means to be 0 across the grade levels. This baseline model of four genre-based factors was based on the findings of Kiili et al. (2023) regarding the factor structure of students’ online credibility evaluation skills among sixth-grade students. Each genre-based factor consisted of three credibility evaluation items (e.g., author expertise of the popular science text, author benevolence of the popular science text, and quality of evidence in the popular science text). However, based on the low factor loading and weak correlations with the other items, we dropped one science newspaper article item (author expertise) from the analysis. Further, in line with Kiili et al. (2023), sixth grade was set as the first in the group order (i.e., the reference grade level). If Model 1 fit the data well, and if all the factor loadings were statistically significant, the configural invariance held. Second, metric invariance (Model 2) was tested by constraining all the factor loadings of the credibility evaluation items to be equal across grade levels. Again, the thresholds of the credibility evaluation items were freely estimated, and the residual variances were fixed to 1. Furthermore, the latent variances were constrained to be 1 at the sixth grade and freed at other levels, but the latent means were still constrained to be 0 across the grade levels. If Model 2 did not differ statistically significantly from Model 1, the metric invariance held. Third, in testing s calar invariance (Model 3), both the factor loadings and the thresholds were set to be equal across the grade levels, and the residual variances were again fixed to 1. The latent variances and the latent means were now both freed at all other grade levels except the sixth grade. If Model 3 did not differ from Model 2, scalar invariance was achieved. Fourth, strict invariance (Model 4) was tested by constraining the factor loadings and thresholds to be equal. Residual variances were now freely estimated across the grade levels, except in the sixth-grade group, in which the residual variances were fixed to 1. In addition, the latent variances and the latent means were again both freed at all other grade levels except the sixth grade. If Model 3 did not differ from Model 4, strict invariance was achieved. The following cutoff criteria were used to indicate a good model fit of all the estimated models: a chi-square (χ 2 ) test with a p-value greater than .05, a root mean square error of approximation (RMSEA) of less than .06, a comparative fit index (CFI) and a Tucker–Lewis index (TLI) of .95 or greater, and a standardized root mean squared residual (SRMR) of less than .08 (Hu & Bentler, 1999). Moreover, only RMSEA and SRMR values above .10 and CFI and TLI values below .90 indicated poor model fit (Kline, 2016). Factorial invariance at each of the four phases was achieved if the chi-square difference (∆χ 2 ) test was not statistically significant ( p > .05), and if the chi-square test value was significant, it meant that invariance was not achieved (Dimitrov, 2010). However, for sample sizes over 300 with equal group sizes, the following criteria of noninvariance are recommended: a decrease of .010 or more in CFI supplemented by an increase of .015 or more in RMSEA (Chen, 2007). Thus, in accordance with previous research, we allowed for small differences in the models based on small changes in other fit indices (Chen, 2007; Kline, 2016). Multivariate Analysis of Covariance of Online Credibility Evaluations Across Grade Levels To answer Research Question 2, concerning the differences in students’ online credibility evaluations across the four grade levels, we utilized MANCOVA. The achieved sufficient level of the above-mentioned factorial invariance made it possible to use the saved factor scores of students’ online credibility evaluation skills in further analyses. The reading order of the online texts was controlled for from the saved-factor scores. In addition, students’ prior topic knowledge and reading fluency scores were controlled for. A mean composite score of prior topic knowledge was formed based on the 10 items. Further, a reading fluency factor was formed based on the two fluency tasks using exploratory factor analysis with principal axis factoring (PROMAX rotation). To identify the factor structure, we relied on the eigenvalue being greater than 1 (Kaiser, 1960) in combination with the extracted communalities being above .30 (Tabachnick & Fidell, 2007–2019). The extracted communalities were above .70 for both reading fluency scores. The partial eta squared ( ηp 2 ) was used as a measure of effect size with the following cut-off values: .010 to .059 (small effect), .060 to .139 (moderate effect), and .140 or higher (large effect) (Cohen, 1988). For pairwise comparisons between grade levels, a Bonferroni correction was applied. Results Descriptive Statistics Descriptive statistics of the students’ online credibility evaluations, prior topic knowledge, and reading fluency are presented in Table 2. Further, Spearman correlations among students’ credibility evaluations are presented in Appendix A. [TABLE 2] Invariance of the Online Credibility Evaluation Task Structure The factorial invariance of the online credibility evaluation task across the grade levels (fourth, sixth, eighth, and 10th grades) was examined by testing configural, metric, scalar, and strict invariance. The final model is presented in Figure 1. Configural invariance (Model 1) showed a similar factor structure across all four grade levels [χ 2 (152) = 276.97, p < .001; RMSEA = .067, CFI = .964, TLI = .948, SRMR = .075]. Although the χ 2 test did not quite reach statistical nonsignificance, the other fit indices indicated acceptable model fit and supported the configural invariance of the online credibility evaluation task. Based on the three credibility evaluation items, four genre-based factors were formed: credibility evaluation of (a) a popular science text; (b) a science newspaper article; (c) a layperson’s blog text; and (d) a commercial text. Next, the metric invariance (i.e., equality of the factor loadings; Model 2) was also confirmed. Although the χ 2 test did not quite reach statistical nonsignificance, the other fit indices indicated an acceptable model fit [χ 2 (173) = 297.17, p < .001; RMSEA = .063, CFI = .965, TLI = .955, SRMR = .080]. A comparison of Model 2 with the less constrained baseline model (Model 1) showed that, although the ∆χ 2 test was significant [χ 2 (21) = 38.24, p = .012], favoring Model 1, improvements in CFI (.001) and RMSEA (-.004) values supported Model 2. These results indicated that the factor loadings could be set as equal across the grade levels. Thus, evaluation skills represented the same genre-based construct across the grade levels. Also, our analyses supported the scalar invariance (i.e., equality of the factor loadings and thresholds of the evaluation items; Model 3). Apart from the χ 2 test, Model 3 had an acceptable fit [χ 2 (227) = 365.48, p < .001; RMSEA = .058, CFI = .960, TLI = .962, SRMR = .083]. A comparison of Model 3 with the less constrained Model 2 showed that although the ∆χ 2 test was again significant [∆χ2(54) = 95.11, p = .001], the stability of the other fit indices (CFI, RMSEA) supported Model 3. This indicates that there were no substantial differences between the grade levels in thresholds, suggesting that students interpreted the task items similarly. Finally, we tested the strict invariance (i.e., equality of residual variances across grade levels) using Model 3 again. Before that, the residual variances were freely estimated in Model 4, and apart from the χ 2 test, the fit indices were acceptable [χ 2 (194) = 327.04, p < .001; RMSEA = .061, CFI = .962, TLI = .957, SRMR = .079]. A comparison of Model 3 with the less constrained Model 4 showed that although the ∆χ 2 test was significant [∆χ 2 (33) = 60.81, p = .002], favoring Model 4, no alarming declines were detected for the other fit indices (CFI, RMSEA). Thus, Model 3 holds, with no significant differences in the residual variances (unexplained variability) of the credibility items across the grade levels. In conclusion, the structure of the online credibility evaluation task functioned similarly across the four grade levels, confirming full factorial invariance and eliminating the need for multigroup analyses. This suggests that the fourth, sixth, eighth, and 10th graders evaluated items on author expertise, author benevolence, and the quality of evidence of the four genre-based factors similarly. As a result, confirmatory factor analysis could be conducted on the entire student sample. Figure 1 presents the final model with an acceptable model fit: χ 2 (45) = 201.45, p < .001, RMSEA = .069, CFI = .950, TLI = .926, SRMR = .056. As shown in Figure 1, significant correlations among the four genre-based factors of students’ credibility evaluation skills were revealed. Positive correlations were found between the credibility evaluation factors of the more credible online texts and between the factors of the less credible online texts. This indicates the students’ ability to distinguish between more and less credible online content. However, negative, albeit weaker, correlations between the factors of more and less credible texts suggested that some students struggled with this differentiation, where high scores in evaluating the more credible online texts accompanied low scores in evaluating the less credible ones, or vice versa. [FIGURE 1] Grade Level Differences in Online Credibility Evaluations On the grade level differences in students’ online credibility evaluations, Table 2 shows that overall, students performed better when evaluating the more credible online texts compared to the less credible ones. Furthermore, Figures B1–B4 in the Appendix B present scatter plots of the students’ online credibility evaluation skills by grade level based on the factor scores. The wide spread of the factor scores indicates a notable variation in the students’ online credibility evaluation skills within each grade level. This variation was particularly evident in the evaluation of less credible online texts. Furthermore, students’ weak credibility evaluation skills were apparent across all grade levels and text genres. The MANCOVA results showed a statistically significant difference between the grade levels in the students’ online credibility evaluation skills [ F (12, 1873) = 4.69, p < .001, Wilks’ Λ = .93, ηp 2 = .026 (small effect)]. The genre-specific examinations showed statistically significant differences between the grade levels in the students’ evaluations of laypersons’ blog text [ F (3, 711) = 13.05, p < .001, ηp 2 = .052 (small effect)] and the commercial text [ F (3, 711) = 14.21, p < .001, ηp 2 = .057 (small effect)]. There were no statistically significant differences between the grade levels in the credibility evaluation of the popular science text and the science newspaper article. The pairwise comparisons presented in Table 3 show that 10th graders performed higher than other students in the credibility evaluation of the laypersons’ blog text and commercial text. In relation to the commercial text, eighth graders also performed higher than fourth graders. [TABLE 3] Discussion This study sought to validate an online credibility evaluation task by measuring students’ evaluation skills across four different grade levels (4 th , 6 th , 8 th , and 10 th grades) from primary to upper secondary education. By verifying the previously observed multidimensional structure of online credibility evaluation tasks (Kiili et al., 2023; Kulju et al., 2024) and demonstrating that the assessment task used worked similarly at each grade level, this study broadens our understanding of how students representing different stages of schooling differ in their online credibility evaluation skills. The Structure of the Online Credibility Evaluation Task Was Similar Across Grades Representing Three Educational Levels In line with our expectations, the genre-based factor structure of the online credibility evaluation task, which assesses students’ evaluation skills in terms of author expertise, author benevolence, and the quality of evidence, was confirmed. Along with text credibility, students’ evaluations also reflected the text genre, consistent with previous research showing that text genres matter when readers evaluate online information (Flanagin & Metzger, 2007; Kiili et al., 2023; List et al., 2017). Furthermore, high positive correlations between the more credible online texts and between the less credible ones align with prior findings (Kiili et al., 2018; Kiili et al., 2023). This suggests that there may be transferable skills for evaluating online texts with similar credibility (i.e., more or less credible online texts), even when the text genres vary. Further, the genre-based factor structure was consistent across the four grade levels. The identified equivalent structure aligns with previous studies that found a similar structure among sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024). As the same structure of online credibility evaluation has been found across different text topics (health effects of sugar, learning styles) and similar text genres, the task design used could be applied across various text topics. This, in turn, would enable a longitudinal examination of students’ skills using a consistent task design while varying the topic. Further research is required to determine whether a genre-based structure remains when texts represent different genres. Finally, the correlations between the more and less credible online texts were negative. This suggests that confirming more credible online information and questioning less credible information are, to some extent, distinctive evaluation skills (Kiili et al., 2018; Kiili et al., 2023), suggesting that both skills need to be practiced. Further, negative correlations suggest that some students scored high when evaluating the more credible online texts but low when evaluating the less credible ones, or vice versa. In the former case, students may have trusted all the text. In the latter case, they may have adopted an overly critical stance toward all online information, impairing their ability to differentiate between more and less credible texts. For example, Hoes et al. (2024) showed that misinformation interventions focusing on fact checking or media literacy can unintentionally increase skepticism toward both accurate and inaccurate content. Person-centered research is required to better understand students’ evaluation patterns. Older Students Performed Higher Than Younger Ones When Questioning Less Credible Online Texts Our expectations regarding differences in students’ online credibility evaluation skills were partly confirmed. While no differences were found between grade levels when evaluating the two more credible online texts, older students performed higher than younger ones when evaluating the two less credible texts. This finding aligns with prior research suggesting that older students may have an advantage, particularly when reading and evaluating contradictory information (Macedo-Rouet et al., 2019; McGrew et al., 2018; Pieschl & Sivyer, 2021; Salmerón et al., 2016). However, it is important to note that, while the differences favoring the older students were observed in the evaluation of the less credible texts, the effect sizes were small ( ηp 2 < .06), suggesting that individual variation plays a significant role in online credibility evaluation. This is further illustrated by the scatter plots in Figures B1–B4 in the Appendix B, which reveal considerable variation within all grade levels, including older students, some of whom still struggle with evaluating the credibility of online texts. As we did not observe any differences between the grade levels in evaluating the more credible online texts, it is possible that students, including primary school students, are more familiar with evaluating more credible online texts than less credible ones. For example, they may be more familiar with the genres of more credible online texts, such as newspaper articles and popular science texts, which are probably more commonly used in classroom instruction than typically less credible genres. Moreover, older students may have more experience discerning the credibility of blogs and commercial texts due to greater exposure and possibly more frequent interactions with these text genres. While we controlled for students’ prior topic knowledge about the health benefits of vitamins, we did not account for their genre knowledge. Future research could investigate how genre knowledge contributes to students’ online credibility evaluation. Overall, the students’ credibility evaluation scores for the less credible online texts were considerably lower than for the more credible ones (see Table 2). This result suggests that students struggle more with evaluating the credibility of less credible online texts, which aligns with previous findings (Kiili et al., 2018; Kiili et al., 2023; Pieschl & Sivyer, 2021; Potocki et al., 2020). Notably, previous research has shown that students, especially younger ones, may further struggle to justify their credibility evaluations (Potocki et al., 2020; Salmerón et al., 2016), regardless of the credibility of the evaluated text (Kiili et al., 2024). Limitations This study has several limitations. The main limitation was the removal of one science newspaper article item (author expertise) because of its low factor loading and weak correlations between the other two items. This was somewhat surprising since the item worked well in previous studies with other text topics (Kiili et al., 2023; Kulju et al., 2024). Despite this, the current task captured students’ online credibility evaluation skills well, as shown by the good model fit and the factorial invariance across the grade levels. Another limitation is the delimitation of text genres. We focused only on four online text genres, but there are other online genres, such as wikitexts or shorter social media posts, that were not covered in this study. Further, the items covered only author expertise, author benevolence, and quality of evidence, but none of them asked students to evaluate the text genre. This could be addressed in future studies by incorporating genre-specific items into the task, assisting us in better understanding the role of genre in online credibility evaluation. Additional research is required to assess whether the genre-based structure holds when different genres are incorporated into the task. Finally, this study compared students’ online credibility evaluation skills across four grades and three educational levels using a cross-sectional design. While the observed differences may reflect grade-related trends, the design does not enable conclusions to be made about the development of students’ online credibility evaluation skills from primary to upper secondary education. Theoretical, Methodological, and Instructional Implications This study has important theoretical, methodological, and instructional implications. Theoretically, the results suggest that students’ ability to evaluate the credibility of less credible online texts develops later than their ability to evaluate the credibility of more credible online texts. This may be due to increasing exposure to various types of online texts (see also Abel et al., 2024), the learning of advanced reading strategies, and an increase in different types of prior knowledge (Alexander, 2005), such as topic and genre knowledge. Our initial findings call for longitudinal research to deepen our understanding of the development of online credibility evaluation skills and how individual difference factors contribute to development during different stages. Methodologically, the results demonstrate that the developed grade-invariant task design can validly measure the online credibility evaluation skills of students across different grade and educational levels. A further advantage of the invariance of the task design is that it enables consistent cross-sectional comparisons of students’ online credibility evaluation skills. Additionally, this approach could facilitate longitudinal follow-ups in future studies, allowing for a deeper understanding of how online credibility evaluation skills develop over time. Standardized assessment tools play a crucial role in facilitating consistent measurement practices and providing opportunities to assess students’ skills across educational institutions. Instructionally, the results suggest that online credibility evaluation skills should be taught throughout school. First, the results showed that older students were only slightly better than younger students in evaluating less credible online texts. Thus, there is still room for improvement at the beginning of upper secondary school. Second, the results show that some older students may also struggle with online credibility evaluations. As previous research suggests that younger students do not benefit from inductive learning of online credibility evaluation skills (Abel et al., 2024), the younger the students are, the more explicit the instruction should be. Regardless of grade level, students should have opportunities to evaluate less credible online texts alongside traditional classroom materials, such as textbooks and newspaper articles, so that they can learn to evaluate both more credible and less credible texts. However, the complexities of online credibility evaluation should be revealed to students in a manner appropriate to their developmental level (Kiili & Kulju, 2024). Furthermore, as our results show that text genres play a role in online credibility evaluation, students may benefit from instruction that uses various text genres. It is essential to systematically assess students’ online credibility evaluation skills, for example, annually. This enables teachers to identify students with limited abilities who may no longer be making progress and require tailored instructional support. For these assessments, teachers could use the task design employed in this study. Finally, as teaching online credibility evaluation should be continuous across schooling, teachers can employ varying pedagogical materials and tools, such as games (Barzilai et al., 2023), instructional videos (Anttonen et al., 2024), visualization tools (Barzilai et al., 2021), and contrasting cases (Braasch et al., 2013) to keep their students engaged. References Abel R, Roelle J, Stadtler M (2024) Whom to believe? Fostering source evaluation skills with interleaved presentation of untrustworthy and trustworthy social media sources. Discourse Processes 61(4–5):233–254. https://doi.org/10.1080/0163853X.2024.2339733 Alexander PA (2005) The path to competence: A lifespan developmental perspective on reading. J Lit Res 37(4):413–436. https://doi.org/10.1207/s15548430jlr3704_1 Anmarkrud Ø, Bråten I, Florit E, Mason L (2022) The role of individual differences in sourcing: A systematic review. Educational Psychol Rev 34:749–792. https://doi.org/10.1007/s10648-021-09640-7 Anttonen R, Kiili K, Räikkönen E, Kiili C (2024) Storifying instructional videos on online credibility evaluation: Examining engagement and learning. Comput Hum Behav 161:108385. https://doi.org/10.1016/j.chb.2024.108385 Barzilai S, Mor-Hagani S, Abed F, Tal-Savir D, Goldik NA, Talmon I, Davidow O (2023) Misinformation is contagious: Middle school students learn how to evaluate and share information responsibly through a digital game. Comput Educ 202:104832. https://doi.org/10.1016/j.compedu.2023.104832 Barzilai S, Tal-Savir D, Abed F, Mor-Hagani S, Zohar AR (2021) Mapping multiple documents: From constructing multiple document models to argumentative writing. Read Writ 36:809–847. https://doi.org/10.1007/s11145-021-10208-8 Barzilai S, Thomm E, Shlomi-Elooz T (2020) Dealing with disagreement: The roles of topic familiarity and disagreement explanation in evaluation of conflicting expert claims and sources. Learn Instruction 69:101367. https://doi.org/10.1016/j.learninstruc.2020.101367 Barzilai S, Tzadok E, Eshet-Alkalai Y (2015) Sourcing while reading divergent expert accounts: Pathways from views of knowing to written argumentation. Instr Sci 43:737–766. https://doi.org/10.1007/s11251-015-9359-4 Barzilai S, Zohar A (2012) Epistemic thinking in action: Evaluating and integrating online sources. Cognition Instruction 30(1):39–85. https://doi.org/10.1080/07370008.2011.636495 Baum LA, Danovitch JH, Keil FC (2008) Children’s sensitivity to circular explanations. J Exp Child Psychol 100(2):146–155. https://doi.org/10.1016/j.jecp.2007.10.007 Braasch JL, Bråten I, Strømsø HI, Anmarkrud Ø, Ferguson LE (2013) Promoting secondary school students’ evaluation of source features of multiple documents. Contemp Educ Psychol 38(3):180–195. https://doi.org/10.1016/j.cedpsych.2013.03.003 Bråten I, McCrudden MT, Lund S, Brante E, E. W., Strømsø HI (2018) Task-oriented learning with multiple documents: Effects of topic familiarity, author expertise, and content relevance on document selection, processing, and use. Reading Res Q 53(3):345–365. https://doi.org/10.1002/rrq.197 Chen FF (2007) Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equation Modeling: Multidisciplinary J 14(3):464–504. https://doi.org/10.1080/10705510701301834 Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum Coiro J, Coscarelli C, Maykel C, Forzani E (2015) Investigating criteria that seventh graders use to evaluate the quality of online information. J Adolesc Adult Lit 59(3):287–297. https://doi.org/10.1002/jaal.448 Corrigan J, Slomp D (2021) Articulating a sociocognitive construct of writing expertise for the digital age. J Writ Analytics 5:142–196. https://doi.org/10.37514/JWA-J.2021.5.1.05 Dimitrov DM (2010) Testing for factorial invariance in the context of construct validation. Meas Evaluation Couns Dev 43(2):121–149. https://doi.org/10.1177/0748175610373459 Ecker UK, Lewandowsky S, Cook J, Schmid P, Fazio LK, Brashier N, Kendeou P, Vraga EK, Amazeen MA (2022) The psychological drivers of misinformation belief and its resistance to correction. Nat Reviews Psychol 1(1):13–29. https://doi.org/10.1038/s44159-021-00006-y Fendt M, Nistor N, Scheibenzuber C, Artmann B (2023) Sourcing against misinformation: Effects of a scalable lateral reading training based on cognitive apprenticeship. Comput Hum Behav 146:107820. https://doi.org/10.1016/j.chb.2023.107820 Finnish National Agency for Education (2016) National core curriculum for basic education 2014. Finnish National Agency for Education Finnish National Agency for Education (2019) National core curriculum for general upper secondary education 2019. Finnish National Agency for Education Finnish National Agency for Education (2024), September 17 Vocational education. eRequirements. Communication and interaction competence. https://eperusteet.opintopolku.fi/#/en/yhteinentutkinnonosa/tutkinnonosat_106727 Flanagin AJ, Metzger MJ (2007) The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media Soc 9(2):319–342. https://doi.org/10.1177/1461444807075015 Forzani E (2018) How well can students evaluate online science information? Contributions of prior knowledge, gender, socioeconomic status, and offline reading ability. Reading Res Q 53(4):385–390. https://doi.org/10.1002/rrq.218 Forzani E (2020) A three-tiered framework for proactive critical evaluation during online inquiry. J Adolesc Adult Lit 63(4):401–414. https://doi.org/10.1002/jaal.1004 Forzani E, Corrigan J, Kiili C (2022) What does more and less effective internet evaluation entail? Investigating readers’ credibility judgments across content, source, and context. Comput Hum Behav 135:107359. https://doi.org/10.1016/j.chb.2022.107359 Hahnel C, Goldhammer F, Kröhne U, Naumann J (2018) The role of reading skills in the evaluation of online information gathered from search engine environments. Comput Hum Behav 78:223–234. https://doi.org/10.1016/j.chb.2017.10.004 Hoes E, Aitken B, Zhang J, Gackowski T, Wojcieszak M (2024) Prominent misinformation interventions reduce misperceptions but increase scepticism. Nat Hum Behav 8:1545–1553. https://doi.org/10.1038/s41562-024-01884-x Hu LT, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equation Modeling: A Multidisciplinary Journal 6(1):1–55. https://doi.org/10.1080/10705519909540118 Hämäläinen EK, Kiili C, Räikkönen E, Marttunen M (2021) Students’ abilities to evaluate the credibility of online texts: The role of internet-specific epistemic justifications. J Comput Assist Learn 37(5):1409–1422. https://doi.org/10.1111/jcal.12580 Kaiser HF (1960) The application of electronic computers to factor analysis. Educ Psychol Meas 20(1):141–151. https://doi.org/10.1177/001316446002000116 Kammerer Y, Gottschling S, Bråten I (2021) The role of internet-specific justification beliefs in source evaluation and corroboration during web search on an unsettled socio-scientific issue. J Educational Comput Res 59(2):342–378. https://doi.org/10.1177/073563312095273 Kanniainen L, Kiili C, Tolvanen A, Aro M, Leppänen PH (2019) Literacy skills and online research and comprehension: Struggling readers face difficulties online. Read Writ 32:2201–2222. https://doi.org/10.1007/s11145-019-09944-9 Kanniainen L, Kiili C, Tolvanen A, Utriainen J, Aro M, Leu DJ, Leppänen PH (2022) Online research and comprehension performance profiles among sixth-grade students, including those with reading difficulties and/or attention and executive function difficulties. Reading Res Q 57(4):1213–1235. https://doi.org/10.1002/rrq.463 Kiili C, Forzani E, Brante EW, Räikkönen E, Marttunen M (2021) Sourcing on the Internet: Examining the relations among different phases of online inquiry. Computers Educ Open 2:100037. https://doi.org/10.1016/j.caeo.2021.100037 Kiili C, Kulju P (2024) Empowering teachers to educate critical online readers. In: Tarchi C (ed) Critical reading of digital texts: The Emile project Empowering schools in self-regulation of media and information literacy processes, vol 89–107. Hogrefe, pp 129–133 Kiili C, Leu DJ, Utriainen J, Coiro J, Kanniainen L, Tolvanen A, Lohvansuu K, Leppänen PH (2018) Reading to learn from online information: Modeling the factor structure. J Lit Res 50(3):304–334. https://doi.org/10.1177/1086296X18784640 Kiili C, Räikkönen E, Bråten I, Strømsø HI, Hagerman MS (2023) Examining the structure of credibility evaluation when sixth graders read online texts. J Comput Assist Learn 39(3):954–969. https://doi.org/10.1111/jcal.12779 Kiili C, Strømsø HI, Bråten I, Ruotsalainen J, Räikkönen E (2024) Reading comprehension skills and prior topic knowledge serve as resources when adolescents justify the credibility of multiple online texts. Read Psychol 45(7):662–689. https://doi.org/10.1080/02702711.2024.2351485 Kline RB (2016) Principles and practice of structural equation modeling, 4th edn. Guilford Press Kintsch W (1998) Comprehension: A paradigm for cognition. Cambridge University Press Kulju P, Hämäläinen E, Mäkinen M, Räikkönen E, Kiili C (under review / preprint). Pre-service teachers evaluating online texts about learning styles: There is room for improvement in justifying the credibility. https://doi.org/10.31219/osf.io/3zwk8 Lerkkanen M-K, Eklund K, Löytynoja H, Aro M, Poikkeus A-M (2018) YKÄ—Luku- ja kirjoitustaidon arviointimenetelmä yläkouluun [YKÄ—Reading test for lower secondary school]. Niilo Mäki Instituutti Leu DJ, Forzani E, Rhoads C, Maykel C, Kennedy C, Timbrell N (2015) The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Res Q 50(1):37–59. https://doi.org/10.1002/rrq.85 Leu DJ, Kinzer CK, Coiro J, Castek J, Henry LA (2019) New Literacies: A dual level theory of the changing nature of literacy, instruction, and assessment. In: Alvermann DE, Unrau NJ, Sailors M, Ruddell RB (eds) Theoretical Models and Processes of Literacy, 7th edn. Taylor & Francis, pp 319–346. https://doi.org/10.4324/9781315110592 List A, Alexander PA, Stephens LA (2017) Trust but verify: Examining the association between students' sourcing behaviors and ratings of text trustworthiness. Discourse Processes 54(2):83–104. https://doi.org/10.1080/0163853X.2016.1174654 Macedo-Rouet M, Braasch JL, Britt MA, Rouet JF (2013) Teaching fourth and fifth graders to evaluate information sources during text comprehension. Cognition Instruction 31(2):204–226. https://doi.org/10.1080/07370008.2013.769995 Macedo-Rouet M, Potocki A, Scharrer L, Ros C, Stadtler M, Salmerón L, Rouet JF (2019) How good is this page? Benefits and limits of prompting on adolescents’ evaluation of web information quality. Reading Res Q 54(3):299–321. https://doi.org/10.1002/rrq.241 Macedo-Rouet M, Salmerón L, Ros C, Pérez A, Stadtler M, Rouet JF (2020) Are frequent users of social network sites good information evaluators? An investigation of adolescents’ sourcing abilities (¿ Son los usuarios frecuentes de las redes sociales evaluadores competentes? Un estudio de las habilidades de los adolescentes para identificar, evaluar y hacer uso de las fuentes). J Study Educ Dev 43(1):101–138. https://doi.org/10.1080/02103702.2019.1690849 McGrew S, Breakstone J, Ortega T, Smith M, Wineburg S (2018) Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory Res social Educ 46(2):165–193. https://doi.org/10.1080/00933104.2017.1416320 McNeish D (2018) Thanks coefficient alpha, we’ll take it from here. Psychol Methods 23(3):412. https://doi.org/10.1037/met0000144 Muthén LK, Muthén BO (1998–2017) Mplus user’s guide. (8th ed.). Muthén & Muthén Pérez A, Potocki A, Stadtler M, Macedo-Rouet M, Paul J, Salmerón L, Rouet JF (2018) Fostering teenagers' assessment of information reliability: Effects of a classroom intervention focused on critical source dimensions. Learn Instruction 58:53–64 Pieschl S, Sivyer D (2021) Secondary students' epistemic thinking and year as predictors of critical source evaluation of internet blogs. Comput Educ 160:104038. https://doi.org/10.1016/j.compedu.2020.104038 Potocki A, de Pereyra G, Ros C, Macedo-Rouet M, Stadtler M, Salmeron L, Rouet JF (2020) The development of source evaluation skills during adolescence: Exploring different levels of source processing and their relationships. J Study Educ Dev 43(1):19–59. https://doi.org/10.1080/02103702.2019. 1690848 Putnick DL, Bornstein MH (2016) Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev Rev 41:71–90. https://doi.org/10.1016/j.dr.2016.06.004 Rocha YM, de Moura GA, Desidério GA, de Oliveira CH, Lourenço FD, de Nicolete F, L. D (2021) The impact of fake news on social media and its influence on health during the COVID-19 pandemic: A systematic review. J Public Health 31:1007–1016. https://doi.org/10.1007/s10389-021-01658-z Salmerón L, Macedo-Rouet M, Rouet JF (2016) Multiple viewpoints increase students' attention to source features in social question and answer forum messages. J Association Inform Sci Technol 67(10):2404–2419. https://doi.org/10.1002/asi.23585 Smahel D, Machackova H, Mascheroni G, Dedkova L, Staksrud E, Ólafsson K, Livingstone S, Hasebrink U (2020) EU Kids Online 2020: Survey results from 19 countries. EU Kids Online. https://doi.org/10.21953/lse.47fdeqj01ofo Stadtler M, Bromme R (2014) The content–source integration model: A taxonomic description of how readers comprehend conflicting scientific information. In: Rapp DN, Braasch JLG (eds) Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences. MIT Press, pp 379–402 Svedholm-Häkkinen AM, Forzani E, Coiro J, Kiili C (2025) Online credibility evaluation skills in upper secondary students: The role of grade level, argument evaluation, and analytic thinking dispositions. Learning Individual Differences 118:102640. https://doi.org/10.1016/j.lindif.2025.102640 Tabachnick BG, Fidell LS (2007–2019) Using multivariate statistics (7th edition). Pearson Taber KS (2018) The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ 48(6):1273–1296. https://doi.org/10.1007/s11165-016-9602-2 Thomm E, Bromme R (2016) How source information shapes lay interpretations of science conflicts: Interplay between sourcing, conflict explanation, source evaluation, and claim evaluation. Read Writ 29:1629–1652. https://doi.org/10.1007/s11145-016-9638-8 van der Eem M, van Driet J, Brand-Gruwel S, van Boxtel C (2024) Students evaluating the trustworthiness of historical sources and internet sources: A comparison. Br Edu Res J 1–22. https://doi.org/10.1002/berj.4095 Tables Table 1 to 3 are available in the Supplementary Files section. Additional Declarations The authors declare no competing interests. Supplementary Files AppendixA.docx Appendix A AppendixB.docx Appendix B Tables.docx Cite Share Download PDF Status: Posted Version 1 posted You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-7815537","acceptedTermsAndConditions":true,"allowDirectSubmit":true,"archivedVersions":[],"articleType":"Research Article","associatedPublications":[],"authors":[{"id":527031624,"identity":"9bbb2a56-ceb0-436c-8395-e892a062a51e","order_by":0,"name":"Laura Kanniainen","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA90lEQVRIiWNgGAWjYBACPgiVAMSMDWCKH8KX48GlhQ1Di2QbmG9MjBYoMDgGpoxxOoyN/YzZhx8MafLm0s2NHx7usMkzvt9j9uABg4EMTi08OcYzexhyDHfOOdgskXgmrdjsGI+5QQKDAR6H5Rgz8DBUMG64kdggkdh2OHHbMR4ziQSGP7i18L8xZvzDUGEP1NL8I7Htf+LmNrAWPLZI5Bgz8zDkJAK1tAFtOZC4gY2glmfFzDIGackgLRaJbcmJM46llUkkGODWws+fvJnxTUWy7YYb6Y9v/myzS+xvPrxN8keFgT0uLRBgQITIKBgFo2AUjAISAAADn06FT/bAVwAAAABJRU5ErkJggg==","orcid":"https://orcid.org/0000-0001-6651-6534","institution":"Faculty of Education and Culture, Tampere University","correspondingAuthor":true,"prefix":"","firstName":"Laura","middleName":"","lastName":"Kanniainen","suffix":""},{"id":527031625,"identity":"ea883923-906f-4056-8220-c54fa3780838","order_by":1,"name":"Asko Tolvanen","email":"","orcid":"https://orcid.org/0000-0001-6430-8897","institution":"Faculty of Education and Psychology, University of Jyväskylä","correspondingAuthor":false,"prefix":"","firstName":"Asko","middleName":"","lastName":"Tolvanen","suffix":""},{"id":527031626,"identity":"a0ee4159-0f06-4d5d-bee7-0362c7a46a93","order_by":2,"name":"Eija Räikkönen","email":"","orcid":"https://orcid.org/0000-0003-4450-9178","institution":"Faculty of Education and Psychology, University of Jyväskylä","correspondingAuthor":false,"prefix":"","firstName":"Eija","middleName":"","lastName":"Räikkönen","suffix":""},{"id":527031627,"identity":"1f7df486-8bdf-4280-b3b6-d489215b91b7","order_by":3,"name":"Riikka Anttonen","email":"","orcid":"https://orcid.org/0000-0002-6474-5621","institution":"Faculty of Education and Culture, Tampere University","correspondingAuthor":false,"prefix":"","firstName":"Riikka","middleName":"","lastName":"Anttonen","suffix":""},{"id":527031628,"identity":"7821df03-375d-44f9-88af-3356f1ce57fb","order_by":4,"name":"Panayiota Kendeou","email":"","orcid":"https://orcid.org/0000-0002-0392-7659","institution":"Department of Educational Psychology, University of Minnesota","correspondingAuthor":false,"prefix":"","firstName":"Panayiota","middleName":"","lastName":"Kendeou","suffix":""},{"id":527031629,"identity":"61da7ab8-a076-45c2-9717-62fad83a7559","order_by":5,"name":"Carita Kiili","email":"","orcid":"https://orcid.org/0000-0001-9189-4094","institution":"Faculty of Education and Culture, Tampere University","correspondingAuthor":false,"prefix":"","firstName":"Carita","middleName":"","lastName":"Kiili","suffix":""}],"badges":[],"createdAt":"2025-10-09 09:16:33","currentVersionCode":1,"declarations":{"humanSubjects":true,"vertebrateSubjects":false,"conflictsOfInterestStatement":false,"humanSubjectEthicalGuidelines":true,"humanSubjectConsent":true,"humanSubjectClinicalTrial":false,"humanSubjectCaseReport":false,"vertebrateSubjectEthicalGuidelines":false},"doi":"10.21203/rs.3.rs-7815537/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-7815537/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":93208958,"identity":"6fcf92fa-c8a7-43ad-8461-3e028f814339","added_by":"auto","created_at":"2025-10-10 08:37:26","extension":"docx","order_by":0,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":81232,"visible":true,"origin":"","legend":"","description":"","filename":"Manuscript.docx","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/1e24f480f94827aeaa3cfecc.docx"},{"id":93210216,"identity":"802ad62e-dd2d-4aa5-8ebe-34644f3a6674","added_by":"auto","created_at":"2025-10-10 08:45:26","extension":"json","order_by":1,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":342,"visible":true,"origin":"","legend":"","description":"","filename":"rs7815537.json","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/da84782f5d5016676ec6e051.json"},{"id":93208962,"identity":"64991a01-3ddd-42e9-a18a-372c866b73bd","added_by":"auto","created_at":"2025-10-10 08:37:26","extension":"xml","order_by":2,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":210839,"visible":true,"origin":"","legend":"","description":"","filename":"rs78155370enriched.xml","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/60b2d1951f309db7950b99c0.xml"},{"id":93208965,"identity":"9cfa372f-fe6f-4620-a559-045def669f33","added_by":"auto","created_at":"2025-10-10 08:37:27","extension":"xml","order_by":3,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":208492,"visible":true,"origin":"","legend":"","description":"","filename":"rs78155370structuring.xml","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/54e045a05bba27a759417ddf.xml"},{"id":93208963,"identity":"52639d67-b0c7-4c27-a7b8-6ea7346455ba","added_by":"auto","created_at":"2025-10-10 08:37:27","extension":"html","order_by":4,"title":"","display":"","copyAsset":false,"role":"acdc-reference","size":217753,"visible":true,"origin":"","legend":"","description":"","filename":"earlyproof.html","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/1437ae2b7cb382eea7df9686.html"},{"id":93208960,"identity":"ce7456a2-5459-4355-9e31-098ed612b203","added_by":"auto","created_at":"2025-10-10 08:37:26","extension":"png","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":254887,"visible":true,"origin":"","legend":"\u003cp\u003eSee image above for figure legend\u003c/p\u003e","description":"","filename":"Figure11.png","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/897a819e9ad8bea878db0290.png"},{"id":93210580,"identity":"64417a6d-efe4-4bb1-a0f5-b7bd4bd892a7","added_by":"auto","created_at":"2025-10-10 08:53:27","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1111164,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/f102a30d-c2b1-4e91-bb3d-217e81bc7549.pdf"},{"id":93208956,"identity":"52a98c1e-fdc0-4ba0-9af5-b9e79dd71068","added_by":"auto","created_at":"2025-10-10 08:37:26","extension":"docx","order_by":1,"title":"","display":"","copyAsset":false,"role":"supplement","size":33608,"visible":true,"origin":"","legend":"\u003cp\u003eAppendix A\u003c/p\u003e","description":"","filename":"AppendixA.docx","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/913744549735b24ec8a1e428.docx"},{"id":93208961,"identity":"3e1351be-6d0b-4868-a658-83b207b052f1","added_by":"auto","created_at":"2025-10-10 08:37:26","extension":"docx","order_by":2,"title":"","display":"","copyAsset":false,"role":"supplement","size":345758,"visible":true,"origin":"","legend":"\u003cp\u003eAppendix B\u003c/p\u003e","description":"","filename":"AppendixB.docx","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/030921593299cb2ad93e42f7.docx"},{"id":93210217,"identity":"406e7c3f-3afc-49b1-a2c6-f1344e92e052","added_by":"auto","created_at":"2025-10-10 08:45:26","extension":"docx","order_by":3,"title":"","display":"","copyAsset":false,"role":"supplement","size":44602,"visible":true,"origin":"","legend":"","description":"","filename":"Tables.docx","url":"https://assets-eu.researchsquare.com/files/rs-7815537/v1/3a1479267b0a4642262086a8.docx"}],"financialInterests":"The authors declare no competing interests.","formattedTitle":"\u003cp\u003eStudents’ Online Credibility Evaluation Skills Across Four Grades Representing Three Educational Levels: Factorial Invariance and Cross-Sectional Comparisons\u003c/p\u003e","fulltext":[{"header":"Introduction","content":"\u003cp\u003eIn today\u0026rsquo;s digital landscape, children and adolescents are exposed to a wide range of online content, including articles, news, blogs, social media posts, and advertisements (Smahel et al., 2020). While online content provides opportunities for learning and connecting, it also poses a significant risk of encountering misinformation (Ecker et al., 2022). Given the prevalence of inaccurate online information, exemplified by the recent \u0026ldquo;infodemic\u0026rdquo; (Rocha et al., 2021),\u0026nbsp;students must be equipped with sufficient skills to assess the quality of online information, which we call online credibility evaluation. To tailor instruction to appropriately support the development of students\u0026rsquo; online credibility evaluation skills, it is necessary to understand the level of students\u0026rsquo; skills at different stages of their schooling. Although there is a substantial body of research on students\u0026rsquo; online credibility evaluation skills, it is limited in two important respects.\u003c/p\u003e\n\u003cp\u003eFirst, previous research has employed various measures to assess students\u0026rsquo; credibility evaluation skills, such as printed web-alike materials (e.g., Macedo-Rouet et al., 2019; P\u0026eacute;rez et al., 2018), researcher-designed online texts (Kiili et al., 2023), and authentic online texts (Barzilai \u0026amp; Zohar, 2012; Coiro et al., 2015; H\u0026auml;m\u0026auml;l\u0026auml;inen et al., 2021). In addition, students have indicated their credibility evaluations with different items, such as multiple choice, rating scales, ranking and open items, or a combination. These differences in measurement instruments make it challenging to compare findings on students\u0026rsquo; credibility evaluation skills across studies and interpret them across different grade levels. Second, studies cross-sectionally examining credibility evaluation skills across the different grade levels have been conducted without considering the factorial invariance of measures used to assess those skills, leaving it unclear whether the measures used assessed the same construct across different grade levels (Macedo-Roeut et al., 2013; 2019; McGrew et al., 2018; Pieschl \u0026amp; Sivyer, 2021; Potocki et al., 2020; Salmer\u0026oacute;n et al., 2016).\u003c/p\u003e\n\u003cp\u003eTo address these issues, this study has two main aims. The first aim is to examine whether the previously identified structure of students\u0026rsquo; online credibility evaluation skills (Kiili et al., 2023) is consistent across four grade levels, from primary to upper secondary education, and to validate an online credibility evaluation assessment task by testing its factorial invariance across these grade levels. The second aim is to investigate how students\u0026rsquo; online credibility evaluation skills vary across the four grades representing three education levels. Thus, the validation of this newly developed measure will not only offer a methodological contribution but also shed light on students\u0026rsquo; online credibility evaluation skills across grade levels, from primary to upper secondary education, offering valuable knowledge for designing effective instruction.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTheoretical Framing of Online Credibility Evaluation\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe draw on several theoretical frameworks that are central to our exploration of students\u0026rsquo; online credibility evaluation skills. Since readers often encounter multiple online texts of varying quality, the theory of \u003cem\u003eonline research and comprehension\u0026nbsp;\u003c/em\u003e(Leu et al., 2015; 2019) underscores the evaluation of information as an essential component of reading online. Evaluation, which can concern relevance and credibility, may occur in all phases of online research and comprehension (see Kiili et al., 2021, for sourcing). While this theory situates evaluation as a vital part of reading online, other models or frameworks describe how readers evaluate credibility when confronted with conflicting information offline or online.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAccording to the\u003cem\u003e\u0026nbsp;content\u0026ndash;source integration model\u003c/em\u003e (Stadtler \u0026amp; Bromme, 2014), readers can use first- and second-hand evaluation to resolve conflicts when encountering contradictory scientific claims in multiple texts. Readers can employ first-hand evaluation to judge the veracity or accuracy of knowledge claims (i.e., text content) by relying on prior knowledge. They can also employ second-hand evaluation to judge the trustworthiness of information sources, such as the text\u0026rsquo;s author or publisher. When readers encounter familiar topics, they tend to rely on first-hand evaluation, but with less familiar topics, they tend to emphasize second-hand evaluation (e.g., Br\u0026aring;ten et al., 2018).\u003c/p\u003e\n\u003cp\u003eThe \u003cem\u003ebidirectional model of first- and second-hand evaluation strategies\u0026nbsp;\u003c/em\u003e(Barzilai et al., 2020) extends the content\u0026ndash;source integration model by demonstrating that readers employ multiple strategies that can impact content and source evaluation in a bidirectional manner. This model differentiates three first-hand evaluation strategies. First, knowledge-based strategies refer to the evaluation of knowledge claims against one\u0026rsquo;s existing knowledge or beliefs. Second, discourse-based strategies involve evaluating the grounding of claims, including the logic of reasoning and the quality of evidence. For instance, research has shown that even by the time children reach second grade, they begin to understand the flawed nature of circular reasoning in arguments and explanations (Baum et al., 2008). Third, corroboration refers to validating claims with information available from other credible resources.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSecond-hand evaluation strategies focus on evaluating author expertise and benevolence (Barzilai et al., 2020; Stadtler \u0026amp; Bromme, 2014). Author expertise reflects competence in sharing accurate information, while benevolence refers to authors acting in readers\u0026rsquo; interests without pursuing personal gains (P\u0026eacute;rez et al., 2018; Thomm \u0026amp; Bromme, 2016). Readers may question author benevolence if, for example, they identify author commercial interests or one-sided argumentation aligned with the author\u0026rsquo;s personal aims (Kiili et al., 2023). In the present study, models depicting first- and second-hand evaluation informed our task design. We incorporate conflicting texts into our online credibility evaluation task and assess students\u0026rsquo; abilities to evaluate author expertise, author benevolence, and the quality of evidence of these texts.\u003c/p\u003e\n\u003cp\u003eBuilding on the concepts of first- and second-hand evaluation, the \u003cem\u003ecritical online resource evaluation framework\u003c/em\u003e (Forzani, 2020; Forzani et al., 2022) focuses on credibility evaluation on the internet and introduces an additional layer, tertiary evaluation. In tertiary evaluation, readers can consider how the context (i.e., the temporal, social, and political settings of the text) in which the content is presented reflects the text\u0026rsquo;s credibility. One available cue for tertiary evaluation is the text genre (Flanagin \u0026amp; Metzger, 2007; List et al., 2017). Readers can use their online genre knowledge, such as knowledge of a specific genre\u0026rsquo;s organizational structures, conventions, and publication practices, to evaluate the context\u0026rsquo;s trustworthiness (Corrigan \u0026amp; Slomp, 2021). This additional layer is considered in our text design because the designed texts represented various online genres.\u003c/p\u003e\n\u003cp\u003eFinally, previous research has shown that readers\u0026rsquo; ability to evaluate online information may be a multidimensional construct (Kanniainen et al., 2022; Kiili et al., 2023; Kulju et al., 2024). For instance, Kiili et al. (2018, 2023) found that students needed different online evaluation skills to confirm the more credible online texts and to question the less credible ones. This was also evident with a person-centered approach, in which a latent profile analysis revealed a group of adolescent online readers with particular difficulty in questioning the credibility of online information (Kanniainen et al., 2022). Moreover, Kiili et al. (2023) demonstrated that both text genre and text credibility played a role in students\u0026rsquo; online credibility evaluation. In the present study, we are interested in how students representing different educational levels perform in evaluating more and less credible online texts representing various genres.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eDevelopmental Aspects of Online Credibility Evaluation Skills\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe development of students\u0026rsquo; online credibility evaluation skills can be approached using Alexander\u0026rsquo;s (2005) \u003cem\u003edevelopmental model of reading\u003c/em\u003e,\u003cem\u003e\u0026nbsp;\u003c/em\u003ewhich describes how individuals\u0026rsquo; reading skills develop throughout their lifespan. As readers mature, their strategic processing shifts from superficial to deeper levels of processing. In this developmental path, readers\u0026rsquo; topic and domain knowledge becomes crucial for building competence and expertise in reading. Along with gaining new knowledge during schooling, exposure to diverse texts and perspectives further refines students\u0026rsquo; strategies for evaluating text content and information sources, both of which are essential for successfully evaluating online credibility (Svedholm-H\u0026auml;kkinen et al., 2025).\u003c/p\u003e\n\u003cp\u003eRegarding online credibility evaluation, there is a lack of understanding of how students\u0026rsquo; skills develop during schooling. Namely, most research on students\u0026rsquo; online credibility evaluation skills has focused on primary school (e.g., Barzilai \u0026amp; Zohar, 2012; Kanniainen et al., 2019), secondary school (e.g., Coiro et al., 2015; van der Eem et al., 2024), or university (e.g., Barzilai et al., 2015; Fendt et al., 2023) students. In fact, few studies have compared different grade levels, even cross-sectionally, and those that did derived their findings from relatively small sample sizes (\u003cem\u003en\u003c/em\u003e \u0026lt; 300). \u0026nbsp;\u003c/p\u003e\n\u003cp\u003eFor example, Macedo-Rouet et al. (2013) included fourth and fifth graders in their study (Experiment 1: \u003cem\u003eN\u003c/em\u003e = 98, Experiment 2: \u003cem\u003eN\u0026nbsp;\u003c/em\u003e= 96; ages 9\u0026ndash;11 years). Although they found that fifth graders performed higher than fourth graders in identifying the most knowledgeable source of the texts (Experiment 1), they compared only two grade levels. Salmer\u0026oacute;n et al. (2016) examined students from fifth grade to university level (Experiment 1: \u003cem\u003eN\u003c/em\u003e = 137, Experiment 2: \u003cem\u003eN\u0026nbsp;\u003c/em\u003e= 277; ages 10\u0026ndash;21 years). University students more frequently cited source attributes and relied less frequently on personal opinions to justify their forum post selections than younger students, especially when forum posts contained contradictory information (cf. Experiment 1 vs. Experiment 2).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eSimilarly, Macedo-Rouet et al. (2019) studied sixth to 10th graders (Experiment 1: \u003cem\u003eN\u003c/em\u003e = 57, Experiment: 2: \u003cem\u003eN\u003c/em\u003e = 36; ages 12\u0026ndash;16 years) and found that younger students identified expert sources less spontaneously and used source cues more superficially than older students. Pieschl and Sivyer (2021) found differences between seventh, ninth, and 11th graders (\u003cem\u003eN\u003c/em\u003e = 218; ages 12\u0026ndash;17 years). Ninth and 11th graders were able to distinguish credible blog posts from noncredible ones, while seventh graders still struggled. Potocki et al. (2020) identified similar trends in a study of fifth, seventh, and ninth graders and university students (\u003cem\u003eN\u003c/em\u003e = 245, ages 10\u0026ndash;19 years), observing an increased ability to distinguish between proficient and less proficient authors and a shift toward source-based justifications from content-based ones.\u003c/p\u003e\n\u003cp\u003eTo the best of our knowledge, only one study with a large sample size has examined students\u0026rsquo; online credibility evaluation skills. McGrew et al. (2018) explored credibility evaluations among students from middle school to college (\u003cem\u003eN\u003c/em\u003e = 894, no age range available) and found that students struggled to effectively evaluate the sources and evidence across different grade levels. However, the results were not directly comparable across grade levels because the tasks and implementation methods (paper or online) varied among students from different stages of education.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIt is noteworthy that none of the aforementioned studies tested the factorial invariance of the tasks used in their research designs. In other words, it remains unclear whether the designed tasks consistently measured students\u0026rsquo; credibility evaluation skills across various educational levels and grades. Thus, it cannot be determined whether the findings reflected actual differences in skills since they could have been biased if the tasks failed to measure the same construct across age groups. Therefore, there is a need to validate the same credibility evaluation task across different grade levels to increase the validity of cross-sectional comparisons and to enable future longitudinal studies.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePresent Study\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn this study, we set out to validate an online credibility evaluation task for primary (fourth and sixth graders), lower secondary (eighth graders), and upper secondary (10th graders) school students. This task was previously validated for sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024). We aim to extend this validation and employ the task to compare students\u0026rsquo; online credibility evaluation skills across the four grade levels. In the task, students read and evaluate four texts representing different genres: two genres typically representing more credible online texts (a popular science text and a science newspaper article) and two genres typically representing less credible ones (a layperson\u0026rsquo;s blog text and a commercial text).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eStudents are asked to evaluate the texts\u0026rsquo; credibility regarding three credibility aspects: author expertise, author benevolence, and quality of evidence. Kiili et al. (2023) found that these aspects were divided into factors based on the text genres, with both the first-order factor model (i.e., genre-based model) and the second-order factor model (i.e., the genre-based confirming and questioning model) fitting the data well. We use the former as our baseline model in this study because it provides a clear, stepwise approach to testing invariance.\u003c/p\u003e\n\u003cp\u003eOur first goal is to answer \u003cem\u003eResearch Question 1:\u003c/em\u003e Does an online credibility evaluation task measure students\u0026rsquo; abilities to evaluate online texts representing different genres similarly across the four grade levels? We expect that the students\u0026rsquo; online credibility evaluations concerning the aforementioned aspects of credibility will reflect the four factors according to the text genres. We also anticipate that the genre-based factor structure will be similar across the four grade levels, reflecting findings from earlier studies conducted among sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024).\u003c/p\u003e\n\u003cp\u003eNext, we seek to answer \u003cem\u003eResearch Question 2:\u003c/em\u003e How do students\u0026rsquo; online credibility evaluation skills differ across the four grade levels after controlling for their prior topic knowledge and reading fluency? We expect that older students will perform higher than younger students in their online credibility evaluations across all online text genres. This assumption is based on theoretical considerations of reading development suggesting that older students can engage in deeper strategic processing than younger students (Alexander, 2005). Our assumptions are supported by previous empirical findings suggesting that older students perform higher than younger students, especially when they read contradictory information (Macedo-Rouet et al., 2019; McGrew et al., 2018; Pieschl \u0026amp; Sivyer, 2021). They are also supported by the learning objectives of the Finnish curriculums (Finnish National Agency for Education, 2016, 2019, 2024), which emphasize the teaching of multiliteracies across grade levels and subjects. As students mature, they are exposed to reading and evaluating offline and online texts with increasing levels of complexity, perspectives, and contexts; their online credibility evaluation skills should improve during schooling.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn the analysis, we control for prior topic knowledge due to its theoretically well-established role in reading comprehension (Alexander, 2005; Kintsch, 1998), and previous research has shown that prior knowledge may contribute to online credibility evaluation (Forzani, 2018; Kammerer et al., 2021; Kiili et al., 2024). We also control for students\u0026rsquo; reading fluency, since word-level reading skills can predict online credibility evaluation abilities in both primary and secondary school students (H\u0026auml;m\u0026auml;l\u0026auml;inen et al., 2021; Macedo-Rouet et al., 2013, 2020).\u0026nbsp;\u003c/p\u003e"},{"header":"Methods","content":"\u003cp\u003e\u003cstrong\u003eParticipants and\u0026nbsp;\u003c/strong\u003e\u003cstrong\u003eContext\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eA total of 728 students participated (50.6% girls, 48.6% boys, 0.8%\u0026nbsp;non-binary), of whom 139 were fourth graders (\u003cem\u003eM\u003c/em\u003e = 10.50 years, \u003cem\u003eSD\u003c/em\u003e = 0.31), 198 were sixth graders (\u003cem\u003eM\u003c/em\u003e = 12.57 years, \u003cem\u003eSD\u003c/em\u003e = 0.38), 203 were eighth graders (\u003cem\u003eM\u003c/em\u003e = 14.64 years, \u003cem\u003eSD\u003c/em\u003e = 0.33), and 188 were first-year general and vocational upper secondary school students (\u003cem\u003eM\u003c/em\u003e = 16.90 years, \u003cem\u003eSD\u003c/em\u003e = 0.64; hereafter 10\u003csup\u003eth\u003c/sup\u003e graders). Most of the students\u0026rsquo; (91.5%) home language was Finnish, while 5.0% spoke Finnish and some other language at home. Only 3.1% spoke some language other than Finnish at home. All the students had a sufficiently good language level to study in mainstream education following the national curricula (Finnish National Agency for Education, 2016, 2019, 2024).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eStudents were recruited from 42 classrooms representing seven comprehensive schools and from 12 classrooms whose\u0026nbsp;teaching was organized by one consortium of general/vocational upper secondary education. After receiving an ethical statement from the Ethics Committee of the [BLINDED] Region, schools were recruited through phone or email contact with the school principals during the 2021\u0026ndash;2022 school year. The principals forwarded the recruitment request to teachers. All the classes and students participated voluntarily. Students and their guardians, for those under 15 years, signed written consent forms for participation.\u003c/p\u003e\n\u003cp\u003eIn Finland, compulsory education begins with comprehensive school (grades 1\u0026ndash;9), which includes primary (grades 1\u0026ndash;6) and lower secondary levels (grades 7\u0026ndash;9). After completing these levels, students continue to the upper secondary level, which usually lasts three years (corresponding to grades 10\u0026ndash;12). They can choose between general education and vocational education and training. General education provides a broad academic foundation, leading to the Finnish matriculation examination and qualification for higher education. Vocational education, while offering access to higher education, equips students with essential skills in specific fields. Compulsory education continues until students turn 18 or complete their upper secondary education.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOnline Credibility Evaluation Task\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe online credibility evaluation task was created with the Critical Online Reading Research Environment (Kiili et al., 2023), where researchers can design web-based credibility evaluation tasks.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eOnline Texts\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe task asked students to read and evaluate the credibility of four online texts\u0026nbsp;about the health benefits of vitamins (see Table 1). Since the texts were researcher-designed, we manipulated the text content (accuracy of the main claim and quality of the evidence), the source trustworthiness (authors\u0026rsquo; expertise and benevolence), and the text genre. Two of the texts can be considered more credible (a science newspaper article and a popular science text) and two less credible (a layperson\u0026rsquo;s blog text and a commercial text).\u003c/p\u003e\n\u003cp\u003eThe four texts were similar in length and were grouped into two text pairs, which considered the same subtopic with contradictory main claims. Subtopic 1 concerned vitamin C and the flu (vitamin C prevents / does not prevent the flu) and Subtopic 2 concerned the health benefits of multivitamins (multivitamins have health benefits / multivitamins have no health benefits if a person has a normal, healthy diet). To counterbalance the possible effects of the reading order, the students read the texts in two different orders, which were randomly assigned. Regarding Subtopic 1, Group 1 read the less credible text first and then the more credible text. Regarding Subtopic 2, Group 1 read the more credible text first and then the less credible text. Group 2 read the texts within the subtopics in reverse order.\u003c/p\u003e\n\u003cp\u003e[TABLE 1]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eTask and Items\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;During the task, the students were instructed by Max,\u0026nbsp;an avatar fact-checker, to read and evaluate one text at a time. For each online text, students responded to three credibility evaluation items concerning (a) author expertise, (b) author benevolence, and (c) the quality of evidence supporting the text\u0026rsquo;s main claim.\u003c/p\u003e\n\u003cp\u003eThe students provided their credibility evaluations on a six-point rating scale (e.g., how much the author has expertise in the health effects of vitamins: 1 = hardly at all; 6 = very much), with their evaluations scored as follows. For the more credible texts, students received 2 points for using the top of the scale (5 or 6), 1 point for using the middle scale (3 or 4), and 0 points for using the bottom of the scale (1 or 2). For the less credible texts, the scoring was reversed.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn addition to the credibility evaluation items, the task included identification items (identification of the author and the main claim and evidence) and justification items (justifications for the credibility evaluations). Although these multiple-choice items were not used in this study, the students responded to the identification items before the evaluation items. If a student did not answer the identification items correctly, Max provided the correct answer to ensure that the correct author or evidence had been evaluated.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003ePrior Topic Knowledge\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBefore students were instructed to undertake the credibility evaluation task, they responded to 10 true\u0026ndash;false topic knowledge items about vitamins (e.g., \u003cem\u003eOur body is able to produce vitamin C; Multivitamin pills are healthier than dietary vitamins; Spending time in the sun increases vitamin D\u003c/em\u003e). One point was given for each correct response (max. 10 points). Items were embedded in the task environment and used as a control variable (a mean composite of a student\u0026rsquo;s prior topic knowledge) in the subsequent analyses.\u003c/p\u003e\n\u003cp\u003eSince the prior topic knowledge items measured diverse aspects of vitamins, they violated the unidimensionality assumption required for both Cronbach\u0026rsquo;s alpha and McDonald\u0026rsquo;s omega (McNeish, 2018). Since knowledge tests often assess distinct components rather than a single construct (Taber, 2018), we do not report these reliability measures. The 10 items were validated by two medical experts, who checked and modified them if needed.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eReading Fluency Measures\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe measured the students\u0026rsquo; reading fluency using two standardized, time-limited paper-and-pencil tasks: a word-reading task, and a sentence-reading task (Lerkkanen et al., 2018). The word-reading task consisted of 80 items, each with four words and one picture. The students were instructed to connect the correct picture\u0026ndash;word pair by drawing a line between a word and a picture and given two minutes to complete the task. The score was calculated as the number of correctly connected pairs minus 0.33 times the number of incorrect responses.\u003c/p\u003e\n\u003cp\u003eThe sentence-reading task consisted of 70 items, and in each item, the students read one sentence. The task was to conclude whether the content of the sentence was true or not. The sentences were designed in a way that made this distinction easily noticeable (e.g., true: a cow is an animal, false: a fish lives on land). Again, the students had two minutes to complete the task. The score was the number of correct answers minus the number of incorrect ones.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eA reading fluency factor (see the \u003cem\u003eAnalysis Strategy\u003c/em\u003e section) was formed based on these two tasks. McDonald\u0026rsquo;s reliability coefficient for the two reading fluency tasks was .89. The reading fluency factor was used as a control variable.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eProcedure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eData were collected remotely via Microsoft Teams due to COVID-19 restrictions. For 10\u003csup\u003eth\u003c/sup\u003e graders, data were collected during a 75-minute class, while younger students participated in two 45-minute sessions. Paper-and-pencil tasks were distributed to schools before the Teams sessions. At the beginning of each class, the researcher briefly introduced the structure of the tasks via Teams.\u003c/p\u003e\n\u003cp\u003eFirst, the students completed two paper-and-pencil reading fluency tasks, with instructions provided through a one-minute video. The completed tasks were stored in a locked closet until retrieved by the researcher. Second, the students completed the credibility evaluation task, which was allotted 45 minutes, with an option to use their 15-minute recess. Students watched a 1-minute 49-seconds introductory video on how to log in and navigate the task environment, and then logged in using the given codes. The researcher monitored through student progress via the administrative version of the environment. Students completed the credibility evaluation task with computers/laptops at their own pace.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eDuring the task, the researcher\u0026rsquo;s camera and microphone were off, but communication with the teacher and classroom was maintained using Teams chat. For example, the teacher was informed when the first students finished or was instructed if some students were very slow in the task. After all students finished, the researcher thanked them and the teacher. Average task completion times were: 21:34 (\u003cem\u003eSD\u003c/em\u003e = 06:04) for fourth graders, 21:57 (\u003cem\u003eSD\u003c/em\u003e = 05:29) for sixth graders, 19:34 (\u003cem\u003eSD\u003c/em\u003e = 04:26) for eighth graders, and 14:49 (\u003cem\u003eSD\u003c/em\u003e = 04:38) for 10th graders. Notably, the fourth graders\u0026rsquo; version of the task did not include justification items due to time constraints, resulting in a task completion time similar to that of older students.\u003c/p\u003e\n\u003cp\u003eWithin a month, the students and teachers received feedback. The students received encouraging and constructive feedback from Max, the avatar that guided them through the task. Teachers received class- and student-level feedback about their students\u0026rsquo; performance on all tasks.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eStatistical Analyses\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDescriptive and reliability analyses as well as multivariate analysis of covariance (MANCOVA) were performed using SPSS Statistics 28. Multigroup confirmatory factor analyses were conducted using Mplus Version 8.9 (Muth\u0026eacute;n \u0026amp; Muth\u0026eacute;n, 1998\u0026ndash;2017). Before the actual analyses, we noticed that of the 739 original participants, three had interrupted the credibility evaluation task for random reasons (e.g., a dental appointment). These participants were removed from the final dataset. Also, one adult student from the general/vocational upper secondary level was removed based on their age (37 years; cf. \u003cem\u003eParticipants and Context\u003c/em\u003e section). Seven other participants were removed based on the time spent on the task (\u0026plusmn;3 standard deviations from the average). Finally, as the students were nested within 54 different classrooms, intraclass correlation coefficients (ICCs) were calculated for the credibility evaluation items. The ICCs showed that 2.5%\u0026ndash;11.2% of the variance in the items was explained by differences at the classroom level. We accounted for this by using class as a clustering variable with the COMPLEX option in further analyses in Mplus.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eFactorial Invariance of the Online Credibility Evaluation Task Across Different Grade Levels\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eResearch Question 1 explores the measurement invariance of the factor structure of the online credibility evaluation task, representing different genres, across fourth, sixth, eighth, and 10th graders. Multigroup confirmatory factor analyses with the weighted least square mean and variance adjusted estimator for categorical credibility evaluation variables were used to test factorial invariance. The factorial invariance tests were conducted in four phases: (a) configural invariance, (b) metric invariance, (c) scalar invariance, and (d) strict invariance (Putnick \u0026amp; Bornstein, 2016). In each phase, the factor model of the previous phase served as a reference.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn the first phase, we assessed \u003cem\u003econfigural invariance\u0026nbsp;\u003c/em\u003e(Model 1) by freely estimating factor loadings and thresholds of the credibility evaluation items while fixing the residual variances to 1. Further, the latent variances were constrained to be 1 and the latent means to be 0 across the grade levels. This baseline model of four genre-based factors was based on the findings of Kiili et al. (2023) regarding the factor structure of students\u0026rsquo; online credibility evaluation skills among sixth-grade students. Each genre-based factor consisted of three credibility evaluation items (e.g., author expertise of the popular science text, author benevolence of the popular science text, and quality of evidence in the popular science text). However, based on the low factor loading and weak correlations with the other items, we dropped one science newspaper article item\u0026nbsp;(author expertise) from the analysis. Further,\u0026nbsp;in line with Kiili et al. (2023), sixth grade was set as the first in the group order (i.e., the reference grade level). If Model 1 fit the data well, and if all the factor loadings were statistically significant, the configural invariance held.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Second, \u003cem\u003emetric invariance\u003c/em\u003e (Model 2) was tested by constraining all the factor loadings of the credibility evaluation items to be equal across grade levels. Again, the thresholds of the credibility evaluation items were freely estimated, and the residual variances were fixed to 1. Furthermore, the latent variances were constrained to be 1 at the sixth grade and freed at other levels, but the latent means were still constrained to be 0 across the grade levels. If Model 2 did not differ statistically significantly from Model 1, the metric invariance held.\u003c/p\u003e\n\u003cp\u003eThird, in testing s\u003cem\u003ecalar invariance\u003c/em\u003e (Model 3), both the factor loadings and the thresholds were set to be equal across the grade levels, and the residual variances were again fixed to 1. The latent variances and the latent means were now both freed at all other grade levels except the sixth grade. If Model 3 did not differ from Model 2, scalar invariance was achieved.\u003c/p\u003e\n\u003cp\u003eFourth, \u003cem\u003estrict invariance\u003c/em\u003e (Model 4) was tested by constraining the factor loadings and thresholds to be equal. Residual variances were now freely estimated across the grade levels, except in the sixth-grade group, in which the residual variances were fixed to 1. In addition, the latent variances and the latent means were again both freed at all other grade levels except the sixth grade. If Model 3 did not differ from Model 4, strict invariance was achieved.\u003c/p\u003e\n\u003cp\u003eThe following cutoff criteria were used to indicate a good model fit of all the estimated models: a chi-square (\u0026chi;\u003csup\u003e2\u003c/sup\u003e) test with a p-value greater than .05, a root mean square error of approximation (RMSEA) of less than .06, a comparative fit index (CFI) and a Tucker\u0026ndash;Lewis index (TLI) of .95 or greater, and a standardized root mean squared residual (SRMR) of less than .08 (Hu \u0026amp; Bentler, 1999). Moreover, only RMSEA and SRMR values above .10 and CFI and TLI values below .90 indicated poor model fit (Kline, 2016).\u003c/p\u003e\n\u003cp\u003eFactorial invariance at each of the four phases was achieved if the chi-square difference (∆\u0026chi;\u003csup\u003e2\u003c/sup\u003e) test was not statistically significant (\u003cem\u003ep\u003c/em\u003e \u0026gt; .05), and if the chi-square test value was significant, it meant that invariance was not achieved (Dimitrov, 2010). However, for sample sizes over 300 with equal group sizes, the following criteria of noninvariance are recommended: a decrease of .010 or more in CFI supplemented by an increase of .015 or more in RMSEA (Chen, 2007). Thus, in accordance with previous research, we allowed for small differences in the models based on small changes in other fit indices (Chen, 2007; Kline, 2016).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003e\u003cem\u003eMultivariate Analysis of Covariance of Online Credibility Evaluations Across Grade Levels\u003c/em\u003e\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo answer Research Question 2, concerning the differences in students\u0026rsquo; online credibility evaluations across the four grade levels, we utilized MANCOVA. The achieved sufficient level of the above-mentioned factorial invariance made it possible to use the saved factor scores of students\u0026rsquo; online credibility evaluation skills in further analyses. The reading order of the online texts was controlled for from the saved-factor scores.\u003c/p\u003e\n\u003cp\u003eIn addition, students\u0026rsquo; prior topic knowledge and reading fluency scores were controlled for. A mean composite score of prior topic knowledge was formed based on the 10 items. Further, a reading fluency factor was formed based on the two fluency tasks using exploratory factor analysis with principal axis factoring (PROMAX rotation). To identify the factor structure, we relied on the eigenvalue being greater than 1 (Kaiser, 1960) in combination with the extracted communalities being above .30 (Tabachnick \u0026amp; Fidell, 2007\u0026ndash;2019). The extracted communalities were above .70 for both reading fluency scores.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eThe partial eta squared (\u003cem\u003e\u0026eta;p\u003c/em\u003e\u003csup\u003e2\u003c/sup\u003e) was used as a measure of effect size with the following cut-off values: .010 to .059 (small effect), .060 to .139 (moderate effect), and .140 or higher (large effect) (Cohen, 1988). For pairwise comparisons between grade levels, a Bonferroni correction was applied.\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eDescriptive Statistics\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;Descriptive statistics of the students\u0026rsquo; online credibility evaluations, prior topic knowledge, and reading fluency are presented in Table 2. Further, Spearman correlations among students\u0026rsquo; credibility evaluations are presented in Appendix A.\u003c/p\u003e\n\u003cp\u003e[TABLE 2]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eInvariance of the Online Credibility Evaluation Task Structure\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThe factorial invariance of the online credibility evaluation task across the grade levels (fourth, sixth, eighth, and 10th grades) was examined by testing configural, metric, scalar, and strict invariance. The final model is presented in Figure 1. \u003cem\u003eConfigural invariance\u003c/em\u003e (Model 1) showed a similar factor structure across all four grade levels [\u0026chi;\u003csup\u003e2\u003c/sup\u003e(152) = 276.97, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001; RMSEA = .067, CFI = .964, TLI = .948, SRMR = .075]. Although the \u0026chi;\u003csup\u003e2\u003c/sup\u003e test did not quite reach statistical nonsignificance, the other fit indices indicated acceptable model fit and supported the configural invariance of the online credibility evaluation task. Based on the three credibility evaluation items, four genre-based factors were formed: credibility evaluation of (a) a popular science text; (b) a science newspaper article; (c) a layperson\u0026rsquo;s blog text; and (d) a commercial text.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNext, the \u003cem\u003emetric invariance\u003c/em\u003e (i.e., equality of the factor loadings; Model 2) was also confirmed. Although the \u0026chi;\u003csup\u003e2\u0026nbsp;\u003c/sup\u003etest did not quite reach statistical nonsignificance, the other fit indices indicated an acceptable model fit [\u0026chi;\u003csup\u003e2\u003c/sup\u003e(173) = 297.17, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001; RMSEA = .063, CFI = .965, TLI = .955, SRMR = .080]. A comparison of Model 2 with the less constrained baseline model (Model 1) showed that, although the ∆\u0026chi;\u003csup\u003e2\u0026nbsp;\u003c/sup\u003etest was significant [\u0026chi;\u003csup\u003e2\u003c/sup\u003e(21) = 38.24, \u003cem\u003ep\u003c/em\u003e = .012], favoring Model 1, improvements in CFI (.001) and RMSEA (-.004) values supported Model 2. These results indicated that the factor loadings could be set as equal across the grade levels. Thus, evaluation skills represented the same genre-based construct across the grade levels.\u003c/p\u003e\n\u003cp\u003eAlso, our analyses supported the \u003cem\u003escalar invariance\u003c/em\u003e (i.e., equality of the factor loadings and thresholds of the evaluation items; Model 3). Apart from the \u0026chi;\u003csup\u003e2\u003c/sup\u003e test, Model 3 had an acceptable fit [\u0026chi;\u003csup\u003e2\u003c/sup\u003e(227) = 365.48, \u003cem\u003ep\u0026nbsp;\u003c/em\u003e\u0026lt; .001; RMSEA = .058, CFI = .960, TLI = .962, SRMR = .083]. A comparison of Model 3 with the less constrained Model 2 showed that although the ∆\u0026chi;\u003csup\u003e2\u0026nbsp;\u003c/sup\u003etest was again significant [∆\u0026chi;2(54) = 95.11, \u003cem\u003ep\u0026nbsp;\u003c/em\u003e= .001], the stability of the other fit indices (CFI, RMSEA) supported Model 3. This indicates that there were no substantial differences between the grade levels in thresholds, suggesting that students interpreted the task items similarly.\u003c/p\u003e\n\u003cp\u003eFinally, we tested the \u003cem\u003estrict invariance\u003c/em\u003e (i.e., equality of residual variances across grade levels) using Model 3 again. Before that, the residual variances were freely estimated in Model 4, and apart from the \u0026chi;\u003csup\u003e2\u003c/sup\u003e test, the fit indices were acceptable [\u0026chi;\u003csup\u003e2\u003c/sup\u003e(194) = 327.04, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001; RMSEA = .061, CFI = .962, TLI = .957, SRMR = .079]. A comparison of Model 3 with the less constrained Model 4 showed that although the ∆\u0026chi;\u003csup\u003e2\u003c/sup\u003e test was significant [∆\u0026chi;\u003csup\u003e2\u003c/sup\u003e(33) = 60.81, \u003cem\u003ep\u0026nbsp;\u003c/em\u003e= .002], favoring Model 4, no alarming declines were detected for the other fit indices (CFI, RMSEA). Thus, Model 3 holds, with no significant differences in the residual variances (unexplained variability) of the credibility items across the grade levels.\u003c/p\u003e\n\u003cp\u003eIn conclusion, the structure of the online credibility evaluation task functioned similarly across the four grade levels, confirming full factorial invariance and eliminating the need for multigroup analyses. This suggests that the fourth, sixth, eighth, and 10th graders evaluated items on author expertise, author benevolence, and the quality of evidence of the four genre-based factors similarly. As a result, confirmatory factor analysis could be conducted on the entire student sample. Figure 1 presents the final model with an acceptable model fit: \u0026chi;\u003csup\u003e2\u003c/sup\u003e(45) = 201.45, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001, RMSEA = .069, CFI = .950, TLI = .926, SRMR = .056.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eAs shown in Figure 1, significant correlations among the four genre-based factors of students\u0026rsquo; credibility evaluation skills were revealed. Positive correlations were found between the credibility evaluation factors of the more credible online texts and between the factors of the less credible online texts. This indicates the students\u0026rsquo; ability to distinguish between more and less credible online content. However, negative, albeit weaker, correlations between the factors of more and less credible texts suggested that some students struggled with this differentiation, where high scores in evaluating the more credible online texts accompanied low scores in evaluating the less credible ones, or vice versa.\u003c/p\u003e\n\u003cp\u003e[FIGURE 1]\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eGrade Level Differences in Online Credibility Evaluations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOn the grade level differences in students\u0026rsquo; online credibility evaluations, Table 2 shows that overall, students performed better when evaluating the more credible online texts compared to the less credible ones. Furthermore, Figures B1\u0026ndash;B4 in the Appendix B present scatter plots of the students\u0026rsquo; online credibility evaluation skills by grade level based on the factor scores. The wide spread of the factor scores indicates a notable variation in the students\u0026rsquo; online credibility evaluation skills within each grade level. This variation was particularly evident in the evaluation of less credible online texts. Furthermore, students\u0026rsquo; weak credibility evaluation skills were apparent across all grade levels and text genres.\u003c/p\u003e\n\u003cp\u003eThe MANCOVA results showed a statistically significant difference between the grade levels in the students\u0026rsquo; online credibility evaluation skills [\u003cem\u003eF\u003c/em\u003e(12, 1873) = 4.69, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001, Wilks\u0026rsquo; \u0026Lambda; = .93, \u003cem\u003e\u0026eta;p\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e = .026 (small effect)]. The genre-specific examinations showed statistically significant differences between the grade levels in the students\u0026rsquo; evaluations of laypersons\u0026rsquo; blog text [\u003cem\u003eF\u003c/em\u003e(3, 711) = 13.05, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001, \u003cem\u003e\u0026eta;p\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e = .052 (small effect)] and the commercial text [\u003cem\u003eF\u003c/em\u003e(3, 711) = 14.21, \u003cem\u003ep\u003c/em\u003e \u0026lt; .001, \u003cem\u003e\u0026eta;p\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e = .057 (small effect)]. There were no statistically significant differences between the grade levels in the credibility evaluation of the popular science text and the science newspaper article. The pairwise comparisons presented in Table 3 show that 10th graders performed higher than other students in the credibility evaluation of the laypersons\u0026rsquo; blog text and commercial text. In relation to the commercial text, eighth graders also performed higher than fourth graders.\u003c/p\u003e\n\u003cp\u003e\u0026nbsp;[TABLE 3]\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThis study sought to validate an online credibility evaluation task by measuring students\u0026rsquo; evaluation skills across four different grade levels (4\u003csup\u003eth\u003c/sup\u003e, 6\u003csup\u003eth\u003c/sup\u003e, 8\u003csup\u003eth\u003c/sup\u003e, and 10\u003csup\u003eth\u003c/sup\u003e grades) from primary to upper secondary education. By verifying the previously observed multidimensional structure of online credibility evaluation tasks (Kiili et al., 2023; Kulju et al., 2024) and demonstrating that the assessment task used worked similarly at each grade level, this study broadens our understanding of how students representing different stages of schooling differ in their online credibility evaluation skills.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eThe Structure of the Online Credibility Evaluation Task Was Similar Across Grades Representing Three Educational Levels\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eIn line with our expectations, the genre-based factor structure of the online credibility evaluation task, which assesses students\u0026rsquo; evaluation skills in terms of author expertise, author benevolence, and the quality of evidence, was confirmed. Along with text credibility, students\u0026rsquo; evaluations also reflected the text genre, consistent with previous research showing that text genres matter when readers evaluate online information (Flanagin \u0026amp; Metzger, 2007; Kiili et al., 2023; List et al., 2017). Furthermore, high positive correlations between the more credible online texts and between the less credible ones align with prior findings (Kiili et al., 2018; Kiili et al., 2023). This suggests that there may be transferable skills for evaluating online texts with similar credibility (i.e., more or less credible online texts), even when the text genres vary.\u003c/p\u003e\n\u003cp\u003eFurther, the genre-based factor structure was consistent across the four grade levels. The identified equivalent structure aligns with previous studies that found a similar structure among sixth graders (Kiili et al., 2023) and university students (Kulju et al., 2024). As the same structure of online credibility evaluation has been found across different text topics (health effects of sugar, learning styles) and similar text genres, the task design used could be applied across various text topics. This, in turn, would enable a longitudinal examination of students\u0026rsquo; skills using a consistent task design while varying the topic. Further research is required to determine whether a genre-based structure remains when texts represent different genres.\u003c/p\u003e\n\u003cp\u003eFinally, the correlations between the more and less credible online texts were negative. This suggests that confirming more credible online information and questioning less credible information are, to some extent, distinctive evaluation skills (Kiili et al., 2018; Kiili et al., 2023), suggesting that both skills need to be practiced. Further, negative correlations suggest that some students scored high when evaluating the more credible online texts but low when evaluating the less credible ones, or vice versa. In the former case, students may have trusted all the text. In the latter case, they may have adopted an overly critical stance toward all online information, impairing their ability to differentiate between more and less credible texts. For example, Hoes et al. (2024) showed that misinformation interventions focusing on fact checking or media literacy can unintentionally increase skepticism toward both accurate and inaccurate content. Person-centered research is required to better understand students\u0026rsquo; evaluation patterns.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOlder Students Performed Higher Than Younger Ones When Questioning Less Credible Online Texts\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eOur expectations regarding differences in students\u0026rsquo; online credibility evaluation skills were partly confirmed. While no differences were found between grade levels when evaluating the two more credible online texts, older students performed higher than younger ones when evaluating the two less credible texts. This finding aligns with prior research suggesting that older students may have an advantage, particularly when reading and evaluating contradictory information (Macedo-Rouet et al., 2019; McGrew et al., 2018; Pieschl \u0026amp; Sivyer, 2021; Salmer\u0026oacute;n et al., 2016). However, it is important to note that, while the differences favoring the older students were observed in the evaluation of the less credible texts, the effect sizes were small (\u003cem\u003e\u0026eta;p\u003csup\u003e2\u003c/sup\u003e\u003c/em\u003e \u0026lt; .06), suggesting that individual variation plays a significant role in online credibility evaluation. This is further illustrated by the scatter plots in Figures B1\u0026ndash;B4 in the Appendix B, which reveal considerable variation within all grade levels, including older students, some of whom still struggle with evaluating the credibility of online texts.\u003c/p\u003e\n\u003cp\u003eAs we did not observe any differences between the grade levels in evaluating the more credible online texts, it is possible that students, including primary school students, are more familiar with evaluating more credible online texts than less credible ones. For example, they may be more familiar with the genres of more credible online texts, such as newspaper articles and popular science texts, which are probably more commonly used in classroom instruction than typically less credible genres. Moreover, older students may have more experience discerning the credibility of blogs and commercial texts due to greater exposure and possibly more frequent interactions with these text genres. While we controlled for students\u0026rsquo; prior topic knowledge about the health benefits of vitamins, we did not account for their genre knowledge. Future research could investigate how genre knowledge contributes to students\u0026rsquo; online credibility evaluation.\u003c/p\u003e\n\u003cp\u003eOverall, the students\u0026rsquo; credibility evaluation scores for the less credible online texts were considerably lower than for the more credible ones (see Table 2). This result suggests that students struggle more with evaluating the credibility of less credible online texts, which aligns with previous findings (Kiili et al., 2018; Kiili et al., 2023; Pieschl \u0026amp; Sivyer, 2021; Potocki et al., 2020). Notably, previous research has shown that students, especially younger ones, may further struggle to justify their credibility evaluations (Potocki et al., 2020; Salmer\u0026oacute;n et al., 2016), regardless of the credibility of the evaluated text (Kiili et al., 2024).\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eLimitations\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study has several limitations. The main limitation was the removal of one science newspaper article item\u0026nbsp;(author expertise) because of its low factor loading and weak correlations between the other two items. This was somewhat surprising since the item worked well in previous studies with other text topics (Kiili et al., 2023; Kulju et al., 2024). \u0026nbsp; Despite this, the current task captured students\u0026rsquo; online credibility evaluation skills well, as shown by the good model fit and the factorial invariance across the grade levels.\u003c/p\u003e\n\u003cp\u003eAnother limitation is the delimitation of text genres. We focused only on four online text genres, but there are other online genres, such as wikitexts or shorter social media posts, that were not covered in this study. Further, the items covered only author expertise, author benevolence, and quality of evidence, but none of them asked students to evaluate the text genre. This could be addressed in future studies by incorporating genre-specific items into the task, assisting us in better understanding the role of genre in online credibility evaluation. Additional research is required to assess whether the genre-based structure holds when different genres are incorporated into the task.\u003c/p\u003e\n\u003cp\u003eFinally, this study compared students\u0026rsquo; online credibility evaluation skills across four grades and three educational levels using a cross-sectional design. While the observed differences may reflect grade-related trends, the design does not enable conclusions to be made about the development of students\u0026rsquo; online credibility evaluation skills from primary to upper secondary education.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTheoretical, Methodological, and Instructional Implications\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eThis study has important theoretical, methodological, and instructional implications. Theoretically, the results suggest that students\u0026rsquo; ability to evaluate the credibility of less credible online texts develops later than their ability to evaluate the credibility of more credible online texts. This may be due to increasing exposure to various types of online texts (see also Abel et al., 2024), the learning of advanced reading strategies, and an increase in different types of prior knowledge (Alexander, 2005), such as topic and genre knowledge. Our initial findings call for longitudinal research to deepen our understanding of the development of online credibility evaluation skills and how individual difference factors contribute to development during different stages.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eMethodologically, the results demonstrate that the developed grade-invariant task design can validly measure the online credibility evaluation skills of students across different grade and educational levels. A further advantage of the invariance of the task design is that it enables consistent cross-sectional comparisons of students\u0026rsquo; online credibility evaluation skills. Additionally, this approach could facilitate longitudinal follow-ups in future studies, allowing for a deeper understanding of how online credibility evaluation skills develop over time. Standardized assessment tools play a crucial role in facilitating consistent measurement practices and providing opportunities to assess students\u0026rsquo; skills across educational institutions.\u003c/p\u003e\n\u003cp\u003eInstructionally, the results suggest that online credibility evaluation skills should be taught throughout school. First, the results showed that older students were only slightly better than younger students in evaluating less credible online texts. Thus, there is still room for improvement at the beginning of upper secondary school. Second, the results show that some older students may also struggle with online credibility evaluations. As previous research suggests that younger students do not benefit from inductive learning of online credibility evaluation skills (Abel et al., 2024), the younger the students are, the more explicit the instruction should be.\u003c/p\u003e\n\u003cp\u003eRegardless of grade level, students should have opportunities to evaluate less credible online texts alongside traditional classroom materials, such as textbooks and newspaper articles, so that they can learn to evaluate both more credible and less credible texts. However, the complexities of online credibility evaluation should be revealed to students in a manner appropriate to their developmental level (Kiili \u0026amp; Kulju, 2024). Furthermore, as our results show that text genres play a role in online credibility evaluation, students may benefit from instruction that uses various text genres.\u003c/p\u003e\n\u003cp\u003eIt is essential to systematically assess students\u0026rsquo; online credibility evaluation skills, for example, annually. This enables teachers to identify students with limited abilities who may no longer be making progress and require tailored instructional support. For these assessments, teachers could use the task design employed in this study. Finally, as teaching online credibility evaluation should be continuous across schooling, teachers can employ varying pedagogical materials and tools, such as games (Barzilai et al., 2023), instructional videos (Anttonen et al., 2024), visualization tools (Barzilai et al., 2021), and contrasting cases (Braasch et al., 2013) to keep their students engaged.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\u003cli\u003e\u003cspan\u003eAbel R, Roelle J, Stadtler M (2024) Whom to believe? Fostering source evaluation skills with interleaved presentation of untrustworthy and trustworthy social media sources. Discourse Processes 61(4\u0026ndash;5):233\u0026ndash;254. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/0163853X.2024.2339733\u003c/span\u003e\u003cspan address=\"10.1080/0163853X.2024.2339733\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAlexander PA (2005) The path to competence: A lifespan developmental perspective on reading. J Lit Res 37(4):413\u0026ndash;436. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1207/s15548430jlr3704_1\u003c/span\u003e\u003cspan address=\"10.1207/s15548430jlr3704_1\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAnmarkrud \u0026Oslash;, Br\u0026aring;ten I, Florit E, Mason L (2022) The role of individual differences in sourcing: A systematic review. Educational Psychol Rev 34:749\u0026ndash;792. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10648-021-09640-7\u003c/span\u003e\u003cspan address=\"10.1007/s10648-021-09640-7\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eAnttonen R, Kiili K, R\u0026auml;ikk\u0026ouml;nen E, Kiili C (2024) Storifying instructional videos on online credibility evaluation: Examining engagement and learning. Comput Hum Behav 161:108385. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.chb.2024.108385\u003c/span\u003e\u003cspan address=\"10.1016/j.chb.2024.108385\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarzilai S, Mor-Hagani S, Abed F, Tal-Savir D, Goldik NA, Talmon I, Davidow O (2023) Misinformation is contagious: Middle school students learn how to evaluate and share information responsibly through a digital game. Comput Educ 202:104832. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.compedu.2023.104832\u003c/span\u003e\u003cspan address=\"10.1016/j.compedu.2023.104832\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarzilai S, Tal-Savir D, Abed F, Mor-Hagani S, Zohar AR (2021) Mapping multiple documents: From constructing multiple document models to argumentative writing. Read Writ 36:809\u0026ndash;847. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11145-021-10208-8\u003c/span\u003e\u003cspan address=\"10.1007/s11145-021-10208-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarzilai S, Thomm E, Shlomi-Elooz T (2020) Dealing with disagreement: The roles of topic familiarity and disagreement explanation in evaluation of conflicting expert claims and sources. Learn Instruction 69:101367. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.learninstruc.2020.101367\u003c/span\u003e\u003cspan address=\"10.1016/j.learninstruc.2020.101367\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarzilai S, Tzadok E, Eshet-Alkalai Y (2015) Sourcing while reading divergent expert accounts: Pathways from views of knowing to written argumentation. Instr Sci 43:737\u0026ndash;766. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11251-015-9359-4\u003c/span\u003e\u003cspan address=\"10.1007/s11251-015-9359-4\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBarzilai S, Zohar A (2012) Epistemic thinking in action: Evaluating and integrating online sources. Cognition Instruction 30(1):39\u0026ndash;85. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/07370008.2011.636495\u003c/span\u003e\u003cspan address=\"10.1080/07370008.2011.636495\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBaum LA, Danovitch JH, Keil FC (2008) Children\u0026rsquo;s sensitivity to circular explanations. J Exp Child Psychol 100(2):146\u0026ndash;155. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.jecp.2007.10.007\u003c/span\u003e\u003cspan address=\"10.1016/j.jecp.2007.10.007\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBraasch JL, Br\u0026aring;ten I, Str\u0026oslash;ms\u0026oslash; HI, Anmarkrud \u0026Oslash;, Ferguson LE (2013) Promoting secondary school students\u0026rsquo; evaluation of source features of multiple documents. Contemp Educ Psychol 38(3):180\u0026ndash;195. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.cedpsych.2013.03.003\u003c/span\u003e\u003cspan address=\"10.1016/j.cedpsych.2013.03.003\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eBr\u0026aring;ten I, McCrudden MT, Lund S, Brante E, E. W., Str\u0026oslash;ms\u0026oslash; HI (2018) Task-oriented learning with multiple documents: Effects of topic familiarity, author expertise, and content relevance on document selection, processing, and use. Reading Res Q 53(3):345\u0026ndash;365. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rrq.197\u003c/span\u003e\u003cspan address=\"10.1002/rrq.197\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eChen FF (2007) Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equation Modeling: Multidisciplinary J 14(3):464\u0026ndash;504. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/10705510701301834\u003c/span\u003e\u003cspan address=\"10.1080/10705510701301834\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Lawrence Erlbaum\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCoiro J, Coscarelli C, Maykel C, Forzani E (2015) Investigating criteria that seventh graders use to evaluate the quality of online information. J Adolesc Adult Lit 59(3):287\u0026ndash;297. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/jaal.448\u003c/span\u003e\u003cspan address=\"10.1002/jaal.448\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eCorrigan J, Slomp D (2021) Articulating a sociocognitive construct of writing expertise for the digital age. J Writ Analytics 5:142\u0026ndash;196. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.37514/JWA-J.2021.5.1.05\u003c/span\u003e\u003cspan address=\"10.37514/JWA-J.2021.5.1.05\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eDimitrov DM (2010) Testing for factorial invariance in the context of construct validation. Meas Evaluation Couns Dev 43(2):121\u0026ndash;149. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/0748175610373459\u003c/span\u003e\u003cspan address=\"10.1177/0748175610373459\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eEcker UK, Lewandowsky S, Cook J, Schmid P, Fazio LK, Brashier N, Kendeou P, Vraga EK, Amazeen MA (2022) The psychological drivers of misinformation belief and its resistance to correction. Nat Reviews Psychol 1(1):13\u0026ndash;29. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s44159-021-00006-y\u003c/span\u003e\u003cspan address=\"10.1038/s44159-021-00006-y\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFendt M, Nistor N, Scheibenzuber C, Artmann B (2023) Sourcing against misinformation: Effects of a scalable lateral reading training based on cognitive apprenticeship. Comput Hum Behav 146:107820. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.chb.2023.107820\u003c/span\u003e\u003cspan address=\"10.1016/j.chb.2023.107820\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFinnish National Agency for Education (2016) National core curriculum for basic education 2014. Finnish National Agency for Education\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFinnish National Agency for Education (2019) National core curriculum for general upper secondary education 2019. Finnish National Agency for Education\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFinnish National Agency for Education (2024), September 17 \u003cem\u003eVocational education.\u003c/em\u003e eRequirements. Communication and interaction competence. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://eperusteet.opintopolku.fi/#/en/yhteinentutkinnonosa/tutkinnonosat_106727\u003c/span\u003e\u003cspan address=\"https://eperusteet.opintopolku.fi/#/en/yhteinentutkinnonosa/tutkinnonosat_106727\" targettype=\"URL\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eFlanagin AJ, Metzger MJ (2007) The role of site features, user attributes, and information verification behaviors on the perceived credibility of web-based information. New Media Soc 9(2):319\u0026ndash;342. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/1461444807075015\u003c/span\u003e\u003cspan address=\"10.1177/1461444807075015\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eForzani E (2018) How well can students evaluate online science information? Contributions of prior knowledge, gender, socioeconomic status, and offline reading ability. Reading Res Q 53(4):385\u0026ndash;390. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rrq.218\u003c/span\u003e\u003cspan address=\"10.1002/rrq.218\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eForzani E (2020) A three-tiered framework for proactive critical evaluation during online inquiry. J Adolesc Adult Lit 63(4):401\u0026ndash;414. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/jaal.1004\u003c/span\u003e\u003cspan address=\"10.1002/jaal.1004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eForzani E, Corrigan J, Kiili C (2022) What does more and less effective internet evaluation entail? Investigating readers\u0026rsquo; credibility judgments across content, source, and context. Comput Hum Behav 135:107359. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.chb.2022.107359\u003c/span\u003e\u003cspan address=\"10.1016/j.chb.2022.107359\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHahnel C, Goldhammer F, Kr\u0026ouml;hne U, Naumann J (2018) The role of reading skills in the evaluation of online information gathered from search engine environments. Comput Hum Behav 78:223\u0026ndash;234. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.chb.2017.10.004\u003c/span\u003e\u003cspan address=\"10.1016/j.chb.2017.10.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHoes E, Aitken B, Zhang J, Gackowski T, Wojcieszak M (2024) Prominent misinformation interventions reduce misperceptions but increase scepticism. Nat Hum Behav 8:1545\u0026ndash;1553. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1038/s41562-024-01884-x\u003c/span\u003e\u003cspan address=\"10.1038/s41562-024-01884-x\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eHu LT, Bentler PM (1999) Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equation Modeling: A Multidisciplinary Journal 6(1):1\u0026ndash;55. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/10705519909540118\u003c/span\u003e\u003cspan address=\"10.1080/10705519909540118\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eH\u0026auml;m\u0026auml;l\u0026auml;inen EK, Kiili C, R\u0026auml;ikk\u0026ouml;nen E, Marttunen M (2021) Students\u0026rsquo; abilities to evaluate the credibility of online texts: The role of internet-specific epistemic justifications. J Comput Assist Learn 37(5):1409\u0026ndash;1422. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/jcal.12580\u003c/span\u003e\u003cspan address=\"10.1111/jcal.12580\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKaiser HF (1960) The application of electronic computers to factor analysis. Educ Psychol Meas 20(1):141\u0026ndash;151. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/001316446002000116\u003c/span\u003e\u003cspan address=\"10.1177/001316446002000116\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKammerer Y, Gottschling S, Br\u0026aring;ten I (2021) The role of internet-specific justification beliefs in source evaluation and corroboration during web search on an unsettled socio-scientific issue. J Educational Comput Res 59(2):342\u0026ndash;378. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/073563312095273\u003c/span\u003e\u003cspan address=\"10.1177/073563312095273\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKanniainen L, Kiili C, Tolvanen A, Aro M, Lepp\u0026auml;nen PH (2019) Literacy skills and online research and comprehension: Struggling readers face difficulties online. Read Writ 32:2201\u0026ndash;2222. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11145-019-09944-9\u003c/span\u003e\u003cspan address=\"10.1007/s11145-019-09944-9\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKanniainen L, Kiili C, Tolvanen A, Utriainen J, Aro M, Leu DJ, Lepp\u0026auml;nen PH (2022) Online research and comprehension performance profiles among sixth-grade students, including those with reading difficulties and/or attention and executive function difficulties. Reading Res Q 57(4):1213\u0026ndash;1235. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rrq.463\u003c/span\u003e\u003cspan address=\"10.1002/rrq.463\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKiili C, Forzani E, Brante EW, R\u0026auml;ikk\u0026ouml;nen E, Marttunen M (2021) Sourcing on the Internet: Examining the relations among different phases of online inquiry. Computers Educ Open 2:100037. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.caeo.2021.100037\u003c/span\u003e\u003cspan address=\"10.1016/j.caeo.2021.100037\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKiili C, Kulju P (2024) Empowering teachers to educate critical online readers. In: Tarchi C (ed) Critical reading of digital texts: The Emile project Empowering schools in self-regulation of media and information literacy processes, vol 89\u0026ndash;107. Hogrefe, pp 129\u0026ndash;133\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKiili C, Leu DJ, Utriainen J, Coiro J, Kanniainen L, Tolvanen A, Lohvansuu K, Lepp\u0026auml;nen PH (2018) Reading to learn from online information: Modeling the factor structure. J Lit Res 50(3):304\u0026ndash;334. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1177/1086296X18784640\u003c/span\u003e\u003cspan address=\"10.1177/1086296X18784640\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKiili C, R\u0026auml;ikk\u0026ouml;nen E, Br\u0026aring;ten I, Str\u0026oslash;ms\u0026oslash; HI, Hagerman MS (2023) Examining the structure of credibility evaluation when sixth graders read online texts. J Comput Assist Learn 39(3):954\u0026ndash;969. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1111/jcal.12779\u003c/span\u003e\u003cspan address=\"10.1111/jcal.12779\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKiili C, Str\u0026oslash;ms\u0026oslash; HI, Br\u0026aring;ten I, Ruotsalainen J, R\u0026auml;ikk\u0026ouml;nen E (2024) Reading comprehension skills and prior topic knowledge serve as resources when adolescents justify the credibility of multiple online texts. Read Psychol 45(7):662\u0026ndash;689. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/02702711.2024.2351485\u003c/span\u003e\u003cspan address=\"10.1080/02702711.2024.2351485\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKline RB (2016) Principles and practice of structural equation modeling, 4th edn. Guilford Press\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKintsch W (1998) Comprehension: A paradigm for cognition. Cambridge University Press\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eKulju P, H\u0026auml;m\u0026auml;l\u0026auml;inen E, M\u0026auml;kinen M, R\u0026auml;ikk\u0026ouml;nen E, Kiili C (under review / preprint). Pre-service teachers evaluating online texts about learning styles: There is room for improvement in justifying the credibility. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.31219/osf.io/3zwk8\u003c/span\u003e\u003cspan address=\"10.31219/osf.io/3zwk8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLerkkanen M-K, Eklund K, L\u0026ouml;ytynoja H, Aro M, Poikkeus A-M (2018) YK\u0026Auml;\u0026mdash;Luku- ja kirjoitustaidon arviointimenetelm\u0026auml; yl\u0026auml;kouluun [YK\u0026Auml;\u0026mdash;Reading test for lower secondary school]. Niilo M\u0026auml;ki Instituutti\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLeu DJ, Forzani E, Rhoads C, Maykel C, Kennedy C, Timbrell N (2015) The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Res Q 50(1):37\u0026ndash;59. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rrq.85\u003c/span\u003e\u003cspan address=\"10.1002/rrq.85\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eLeu DJ, Kinzer CK, Coiro J, Castek J, Henry LA (2019) New Literacies: A dual level theory of the changing nature of literacy, instruction, and assessment. In: Alvermann DE, Unrau NJ, Sailors M, Ruddell RB (eds) Theoretical Models and Processes of Literacy, 7th edn. Taylor \u0026amp; Francis, pp 319\u0026ndash;346. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.4324/9781315110592\u003c/span\u003e\u003cspan address=\"10.4324/9781315110592\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eList A, Alexander PA, Stephens LA (2017) Trust but verify: Examining the association between students' sourcing behaviors and ratings of text trustworthiness. Discourse Processes 54(2):83\u0026ndash;104. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/0163853X.2016.1174654\u003c/span\u003e\u003cspan address=\"10.1080/0163853X.2016.1174654\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMacedo-Rouet M, Braasch JL, Britt MA, Rouet JF (2013) Teaching fourth and fifth graders to evaluate information sources during text comprehension. Cognition Instruction 31(2):204\u0026ndash;226. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/07370008.2013.769995\u003c/span\u003e\u003cspan address=\"10.1080/07370008.2013.769995\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMacedo-Rouet M, Potocki A, Scharrer L, Ros C, Stadtler M, Salmer\u0026oacute;n L, Rouet JF (2019) How good is this page? Benefits and limits of prompting on adolescents\u0026rsquo; evaluation of web information quality. Reading Res Q 54(3):299\u0026ndash;321. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/rrq.241\u003c/span\u003e\u003cspan address=\"10.1002/rrq.241\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMacedo-Rouet M, Salmer\u0026oacute;n L, Ros C, P\u0026eacute;rez A, Stadtler M, Rouet JF (2020) Are frequent users of social network sites good information evaluators? An investigation of adolescents\u0026rsquo; sourcing abilities (\u0026iquest; Son los usuarios frecuentes de las redes sociales evaluadores competentes? Un estudio de las habilidades de los adolescentes para identificar, evaluar y hacer uso de las fuentes). J Study Educ Dev 43(1):101\u0026ndash;138. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/02103702.2019.1690849\u003c/span\u003e\u003cspan address=\"10.1080/02103702.2019.1690849\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcGrew S, Breakstone J, Ortega T, Smith M, Wineburg S (2018) Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory Res social Educ 46(2):165\u0026ndash;193. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/00933104.2017.1416320\u003c/span\u003e\u003cspan address=\"10.1080/00933104.2017.1416320\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMcNeish D (2018) Thanks coefficient alpha, we\u0026rsquo;ll take it from here. Psychol Methods 23(3):412. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1037/met0000144\u003c/span\u003e\u003cspan address=\"10.1037/met0000144\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eMuth\u0026eacute;n LK, Muth\u0026eacute;n BO (1998\u0026ndash;2017) \u003cem\u003eMplus user\u0026rsquo;s guide.\u003c/em\u003e (8th ed.). Muth\u0026eacute;n \u0026amp; Muth\u0026eacute;n\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eP\u0026eacute;rez A, Potocki A, Stadtler M, Macedo-Rouet M, Paul J, Salmer\u0026oacute;n L, Rouet JF (2018) Fostering teenagers' assessment of information reliability: Effects of a classroom intervention focused on critical source dimensions. Learn Instruction 58:53\u0026ndash;64\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePieschl S, Sivyer D (2021) Secondary students' epistemic thinking and year as predictors of critical source evaluation of internet blogs. Comput Educ 160:104038. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.compedu.2020.104038\u003c/span\u003e\u003cspan address=\"10.1016/j.compedu.2020.104038\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePotocki A, de Pereyra G, Ros C, Macedo-Rouet M, Stadtler M, Salmeron L, Rouet JF (2020) The development of source evaluation skills during adolescence: Exploring different levels of source processing and their relationships. J Study Educ Dev 43(1):19\u0026ndash;59. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1080/02103702.2019. 1690848\u003c/span\u003e\u003cspan address=\"10.1080/02103702.2019. 1690848\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003ePutnick DL, Bornstein MH (2016) Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Dev Rev 41:71\u0026ndash;90. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.dr.2016.06.004\u003c/span\u003e\u003cspan address=\"10.1016/j.dr.2016.06.004\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eRocha YM, de Moura GA, Desid\u0026eacute;rio GA, de Oliveira CH, Louren\u0026ccedil;o FD, de Nicolete F, L. D (2021) The impact of fake news on social media and its influence on health during the COVID-19 pandemic: A systematic review. J Public Health 31:1007\u0026ndash;1016. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s10389-021-01658-z\u003c/span\u003e\u003cspan address=\"10.1007/s10389-021-01658-z\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSalmer\u0026oacute;n L, Macedo-Rouet M, Rouet JF (2016) Multiple viewpoints increase students' attention to source features in social question and answer forum messages. J Association Inform Sci Technol 67(10):2404\u0026ndash;2419. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/asi.23585\u003c/span\u003e\u003cspan address=\"10.1002/asi.23585\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSmahel D, Machackova H, Mascheroni G, Dedkova L, Staksrud E, \u0026Oacute;lafsson K, Livingstone S, Hasebrink U (2020) \u003cem\u003eEU Kids Online 2020: Survey results from 19 countries.\u003c/em\u003e EU Kids Online. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.21953/lse.47fdeqj01ofo\u003c/span\u003e\u003cspan address=\"10.21953/lse.47fdeqj01ofo\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eStadtler M, Bromme R (2014) The content\u0026ndash;source integration model: A taxonomic description of how readers comprehend conflicting scientific information. In: Rapp DN, Braasch JLG (eds) Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences. MIT Press, pp 379\u0026ndash;402\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eSvedholm-H\u0026auml;kkinen AM, Forzani E, Coiro J, Kiili C (2025) Online credibility evaluation skills in upper secondary students: The role of grade level, argument evaluation, and analytic thinking dispositions. Learning Individual Differences 118:102640. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1016/j.lindif.2025.102640\u003c/span\u003e\u003cspan address=\"10.1016/j.lindif.2025.102640\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTabachnick BG, Fidell LS (2007\u0026ndash;2019) \u003cem\u003eUsing multivariate statistics\u003c/em\u003e (7th edition). Pearson\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eTaber KS (2018) The use of Cronbach\u0026rsquo;s alpha when developing and reporting research instruments in science education. Res Sci Educ 48(6):1273\u0026ndash;1296. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11165-016-9602-2\u003c/span\u003e\u003cspan address=\"10.1007/s11165-016-9602-2\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003eThomm E, Bromme R (2016) How source information shapes lay interpretations of science conflicts: Interplay between sourcing, conflict explanation, source evaluation, and claim evaluation. Read Writ 29:1629\u0026ndash;1652. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1007/s11145-016-9638-8\u003c/span\u003e\u003cspan address=\"10.1007/s11145-016-9638-8\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003cli\u003e\u003cspan\u003evan der Eem M, van Driet J, Brand-Gruwel S, van Boxtel C (2024) Students evaluating the trustworthiness of historical sources and internet sources: A comparison. Br Edu Res J 1\u0026ndash;22. \u003cspan class=\"ExternalRef\"\u003e\u003cspan class=\"RefSource\"\u003ehttps://doi.org/10.1002/berj.4095\u003c/span\u003e\u003cspan address=\"10.1002/berj.4095\" targettype=\"DOI\" class=\"RefTarget\"\u003e\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/li\u003e\u003c/ol\u003e"},{"header":"Tables","content":"\u003cp\u003eTable 1 to 3 are available in the Supplementary Files section.\u003c/p\u003e"}],"fulltextSource":"","fullText":"","funders":[{"identity":"2e00dacb-d285-438d-b19a-bf23ee38b9a8","identifier":"10.13039/501100009047","name":"Strategic Research Council","awardNumber":"335625","order_by":0},{"identity":"72221739-0bb1-4657-8bb3-cb926bf4e244","identifier":"10.13039/501100009047","name":"Strategic Research Council","awardNumber":"358250","order_by":1},{"identity":"f5a724fa-4480-412f-a9f4-1cb469ab8307","identifier":"10.13039/501100002341","name":"Academy of Finland","awardNumber":"324524","order_by":2}],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":true,"hideJournal":true,"highlight":"","institution":"Tampere University","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":true,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true},"keywords":"credibility evaluation, online evaluation, sourcing, digital literacies, factorial invariance","lastPublishedDoi":"10.21203/rs.3.rs-7815537/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-7815537/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eWe sought to validate a web-based online credibility evaluation task across four grade levels and three educational stages. The aim is to provide a robust tool for assessing students\u0026rsquo; online credibility evaluation skills and grade-level differences in skills while controlling for text reading order, prior topic knowledge, and reading fluency. The sample comprised 728 students from primary (fourth and sixth graders), lower secondary (eighth graders), and upper secondary (10th graders) education in Finland. Students evaluated four online texts representing different genres: a popular science text, a science newspaper article, a layperson\u0026rsquo;s blog text, and a commercial text. Each text was evaluated from three perspectives: author expertise, author benevolence, and quality of evidence. The results confirmed the factorial invariance of the task across grade levels, supporting the validity of comparisons between the grade levels. Older students performed higher than younger ones in evaluating less credible online texts (i.e., the blog and commercial text). By offering a theoretically grounded and methodologically robust evaluation approach, our findings contribute to advancing assessment practices in the digital age.\u003c/p\u003e","manuscriptTitle":"Students’ Online Credibility Evaluation Skills Across Four Grades Representing Three Educational Levels: Factorial Invariance and Cross-Sectional Comparisons","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-10-10 08:37:22","doi":"10.21203/rs.3.rs-7815537/v1","editorialEvents":[{"type":"communityComments","content":0}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"researchsquare","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":true,"externalIdentity":"","sideBox":"","snPcode":"","submissionUrl":"/submission","title":"Research Square","twitterHandle":"researchsquare","acdcEnabled":true,"dfaEnabled":false,"editorialSystem":"","reportingPortfolio":"","inReviewEnabled":false,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"304fdf63-5beb-40e7-bcea-7046aa24b71a","owner":[],"postedDate":"October 10th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"posted","subjectAreas":[{"id":56018247,"name":"Educational Psychology"}],"tags":[],"updatedAt":"2025-10-10T08:37:22+00:00","versionOfRecord":[],"versionCreatedAt":"2025-10-10 08:37:22","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-7815537","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-7815537","identity":"rs-7815537","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}