Sample Size and Assessment Length Recommendations for the Diagnostic Status Facet Model

This study investigates the sample size and assessment length requirements of the recently proposed Diagnostic Facet Status Model (DFSM), a polytomous cognitive diagnostic model designed to simultaneously capture goal and intermediate understandings. While prior research established the model using a large sample (N = 4,000) and a long assessment (27 items), practical applications require guidance on whether DFSM can be reliably used with fewer students and shorter assessments. A fully crossed Monte Carlo simulation varying sample sizes (N = 250, 500, 1,000, 4,000) and assessment lengths (J = 9, 18, 27) was conducted where we evaluated item parameter, facet structure, and student latent profile recovery. Results showed that assessment length most strongly influenced item parameter recovery, with at least 18 items necessary to obtain unbiased estimates even for small samples. In contrast, facet sparsity and profile recovery were more sensitive to sample size, requiring at least 500 students for accurate estimation. We provide recommendations for assessment developers and educators on using DFSM in applied contexts.