Bridging the gap between perception and action: the case for neuroimaging, AI and video games

A fundamental goal of computational neuroscience is to account for the implementation of cognitive processes in the brain, yet current models tend to focus on elementary cognitive processes. By contrast, video games have been designed to fully engage players, and require to constantly monitor the state of the game, in parallel to integrating strategic planning, decision making and taking action. While playing video games is hard, recent advances in artificial intelligence (AI) have made it possible to train deep neural networks that reach or even surpass human performance. We discuss challenges and opportunities in training artificial neural networks that could account jointly for human brain activity and behaviour during video game play. We argue that large-scale neuroimaging data may help to constrain the training of artificial networks and open new avenues for research at the intersection of neuroscience and AI.