Prediction of Performance in Standardised Assessments from Computer-Based Formative Assessment Data

Summative assessments (SAs) and formative assessments (FAs) fulfil complementaryfunctions in the educational endeavour. The former measure knowledge acquired at the endof an educational unit in a standardised, high-stakes setting. FAs, by contrast, aim toassess student performance as part of daily classroom activities in order to tailor feedbackand instruction. The use of computer-based FA (CBFA) systems has made it technicallyfeasible to collect unprecedented amounts of longitudinal data objectively and withminimal interference for students, under conditions that more closely resemble real-lifebehaviour. In this paper, we investigated whether and how well FA outcomes can predictSA outcomes in a large sample of children evaluated at different time points duringcompulsory schooling. To this end, we estimated student abilities using Item ResponseTheory and performed a systematic comparison of regression models trained to predict SAabilities on different subsets of features derived from FA abilities and auxiliary variables. Amodel that included mean abilities in different competence domains performed best, andits predictions accounted for a considerable amount of variance, although the proportion ofvariation explained was still below that predicted by past SA measures. The FA featuresshowed specificity in the sense that the most predictive features generally tended tocorrespond to abilities from the same or a similar competence domain as the predicted SAability. Even though CBFA systems implement objective data-collection procedures, weobserved systematic biases in the predictions that would need to be taken intoconsideration when using the models for decision-making.