An Evaluation of Deep Learning Approaches for Factor Analysis of Response and Response Time Data

An important problem in the application of psychometric models is the selection of suitable algorithms for parameter estimation. In a recent publication, Urban and Bauer (Psychometrika 86:1-29, 2021) proposed an estimation algorithm based on deep learning for item parameter estimation for large sample sizes. In this article, we investigate the accuracy of an application of this estimation method for a multidimensional variation of the response time model proposed by Klein Entink, Fox and van der Linden (Psychometrika 74:21-48, 2009) and a joint factor model for responses and response times. The second of these models can be seen as a variation of the joint model for responses and response times of Klein Entink, Fox and van der Linden. Our evaluation focuses on comparing the effects of different sets of hyperparameters on the accuracy of the estimation method. Our results indicate that the deep learning algorithm can be used for an accurate item parameter estimation for these models even in relatively small datasets and is computationally fast in large datasets, but that its accuracy depends on the selection of suitable hyperparameters, such as the learning rate. The evaluated algorithm is freely available in a Python package. We also describe an empirical application of this method to the Amsterdam Chess Test dataset.