Is it Just a Game? Development and Validation of a Deductive Version of Mastermind as a Measure of Reasoning Ability

Building on the long common history of board games and intelligence research, we developed a new deductive reasoning test based on the popular game Mastermind. The research questions of this registered report were: (a) Is a psychometrically sound measurement of the ability to solve Mastermind items possible (i.e., a reliable, uni-dimensional measurement with a good coverage of difficulty)? (b) Is the ability to solve Mastermind items substantially related to other measures of cognitive ability (i.e., matrix test, knowledge test) and need for cognition? (c) Can item difficulty be predicted by the number of colors, positions, premises, and a newly proposed entropy-based index? Based on the results of a pilot study, we developed 30 items and administered them to 351 participants in the preregistered main study using a multiple matrix sampling design. The deductive Mastermind test proved to be (a) a reliable and efficient measure of reasoning across a wide ability range, and (b) showed expectation-consistent patterns for the convergent and divergent measures. (c) The entropy-based index allowed for the prediction of item difficulty to a considerable degree (R² = .47). We discuss the ideas of information theory, including entropy, as constructing principles for the rational test development of reasoning tests.