Unsupervised identification of the internal states that shape natural behavior

Summary Internal states can shape stimulus responses and decision-making, but we lack methods to identify internal states and how they evolve over time. To address this gap, we have developed an unsupervised method to identify internal states from behavioral data, and have applied it to the study of a dynamic social interaction. During courtship, Drosophila melanogaster males pattern their songs using feedback cues from their partner. Our model uncovers three latent states underlying this behavior, and is able to predict the moment-to-moment variation in natural song patterning decisions. These distinct behavioral states correspond to different sensorimotor strategies, each of which is characterized by different mappings from feedback cues to song modes. Using the model, we show that a pair of neurons previously thought to be command neurons for song production are sufficient to drive switching between states. Our results reveal how animals compose behavior from previously unidentified internal states, a necessary step for quantitative descriptions of animal behavior that link environmental cues, internal needs, neuronal activity, and motor outputs.