Using simulations to explore the impact of predictive validity and base rate on classification across univariate and multivariate universal screening approaches

Universal screening to predict students’ risk for reading problems is a foundational component of the Multi-Tiered Systems of Support framework and is required by law in many U.S. states. School or district administrators are tasked with selecting screening assessments that are both technically adequate and feasible given the resources of their local context. One common recommendation is that educational screening assessments should have at least a sensitivity of .9 and a specificity of .8. The present study used simulation methodology to identify the screener-outcome correlation needed to achieve these recommended levels of sensitivity and sensitivity across one- and two-indicator screening batteries and samples with varying base rates of non-proficiency. Results showed that the minimum correlations needed to achieve this recommendation were higher than what is observed in practice and varied across samples with differing base rates. Furthermore, screening assessments with the recommended levels of sensitivity and specificity had high rates of false positive classifications. These results suggest that the practice of issuing specific criteria for the sensitivity and specificity of screeners may be misguided. Implications for the evaluation of the technical adequacy of screening assessments are discussed.