Detecting Differential Item Functioning (DIF) in Multidimensional Item Response Theory (MIRT) Models using Explainable Artificial Intelligence (XAI)

We conducted two studies on using random forest (RF) analysis with explainable artificial intelligence (XAI) to detect differential item functioning (DIF) in multidimensional item response theory (MIRT) models. RF-XAI identifies DIF-items by their importance in predicting group membership from item responses and person parameters. Study 1 examines how test characteristics, namely DIF-item proportion, sample size, test dimensionality, and the RF parameter mtry affect variable importance and detection metrics. High detection rates and low false positive rates occurred in large samples, with low DIF proportions, and medium mtry values. Study 2 compares RF-XAI to Mantel-Haenszel (MH) and logistic regression (LR). RF-XAI slightly outperformed traditional methods in large, multidimensional tests, while MH and LR were more effective in smaller samples and uni-dimensional scales. RF-XAI yielded lower false positive rates in multidimensional designs but had lower detection rates. The results support RF-XAI as a promising tool for enhancing fairness in psychological and educational assessments.