Scoring Two-Part Multiple-Choice Items: An IRT Model Comparison with Implications for Validity Arguments and Process Inference

Two-part multiple-choice items elicit a content decision and a justifying reason, yet the choice of scoring rule that translates the paired responses into a latent-trait estimate is typically made on practical rather than psychometric grounds. We compare eight scoring models -- Part-1-only 2PL, conjunctive 2PL, partial credit, nominal response, mixed sequential, mixture IRT, attribute-group bifactor GPCM, and exploratory diagnostic classification -- fitted to two-part epistemic-knowledge data from N = 299 secondary-school students, mapped to Kane's argument-based validity framework, and stress-tested through a 100-replication simulation across five sample sizes (N = 50, 100, 200, 299, 500) under empirical generating parameters. No single model dominates. Information criteria are reported within comparable response-coding clusters, not across them. Within the polytomous cluster (M3, M7), the attribute-group bifactor generalized partial credit model (GPCM) achieves the lowest empirical AIC but converges in only 22-42% of simulation replications, indicating sample-specific rather than generalizable fit. Across binary recodings, the conjunctive two-parameter logistic (M2) yields lower BIC than the Part-1-only 2PL (M1) in 100% of N = 299 simulation replications, but this comparison is descriptive because the two models operate on different response codings; mixture IRT (M6), fitted on the Part-1 binary matrix, cannot reliably recover latent classes (Adjusted Rand Index near 0). Among non-mixture scoring models, person parameters diverge most consequentially between Part-1-only and conjunctive models (r = .82), demonstrating that scoring choices change score meaning, not merely precision. We propose a conditional framework that aligns each scoring model with an intended interpretation and the validity evidence it requires, treating the scoring rule as a substantive validity inference rather than a technical convenience.