Evaluating youth mathematical ability online and on-site with the Stanford Mental Arithmetic Response Time Evaluation (SMARTE)

Mathematical knowledge is essential to function properly in our modern society. It is therefore crucial for teachers and clinicians to be able to assess learners’ mathematical abilities with validity, reliability, and speed. Here we describe a new tablet-based assessment app designed to measure math fluency, the Stanford Mental Arithmetic Response Time Evaluation (SMARTE) tool. SMARTE consists of three 2-minutes tasks (non-symbolic dot enumeration, math fluency, and math recall). We analyzed data from 6,855 participants (3,262 female, mean age: 12.97 years) who completed SMARTE in year 3 of the Adolescent Brain Cognitive DevelopmentSM study. Because of the COVID-19 crisis, 2,682 participants were tested online while 1,159 participants were tested in person as the original protocol called for. Our analyses revealed that the SMARTE score and sub-scores highly correlated with each other. Overall fluency metrics revealed more problems per unit time were completed in the laboratory setting, yet a host of experimental contrasts that manipulated the cognitive demands across problem types produced equivalent effects across both settings, suggesting these effects are quite robust across variations in study setting.