Benefits of co-learning with an AI agent

The advent of effective machine learning techniques raises the question of how such procedures might benefit human learning. In this paper we study how humans solve a type of classification puzzle—the Game of Hidden Rules (GOHR)—with vs. without the assistance of a “bot” that provides potentially helpful suggestions about how to proceed. In a GOHR game, the learner attempts to sort colored shapes into categories according to a hidden rule that they must discover, for example “red shapes go to bucket #0”, “blue shapes to bucket #1,” etc. In some conditions, a "bot" made suggestions, which the human learner was free to follow or ignore. Even though human learners did not always take the bot’s advice, we found a consistent performance advantage in bot conditions compared to no-bot conditions, meaning that participants solved these problems more quickly when the bot was present than when it was not. This effect was particularly pronounced in lower-performing subjects, while high-performing subjects were relatively unaffected. We also manipulated the learning speed of the bot, and found that the benefit of bot assistance increased with bot "intelligence." Our results demonstrate that bot assistance can be helpful to human learners, and shed some light on the prospects of AI-supported human learning.