Thoughtful faces: inferring internal states across species using facial features

Abstract Animal behaviour is shaped to a large degree by internal cognitive states, but it is unknown whether these states are similar across species. To address this question, we developed a virtual reality setup in which mice and macaques engage in the same naturalistic visual foraging task. We exploited the richness of a wide range of facial features extracted from video recordings during the task, to train a Markov-Switching Linear Regression (MSLR). By doing so, we identified, on a single-trial basis, a set of internal states that reliably predicted when the animals were going to react to the presented stimuli. Even though the model was trained purely on reaction times, it could also predict task outcome, supporting the behavioural relevance of the inferred states. The identified states were comparable between mice and monkeys. Furthermore, each state corresponded to a characteristic pattern of facial features, highlighting the importance of facial expressions as manifestations of internal cognitive states across species.