Computational Prometheus: How reinforcement was stolen, and how to steal it back

There are now tools and techniques in computer science that make building and testing complicated process models of behavior easy. One such approach to process models, “Reinforcement Learning,” has found a second home in neuroscience. When fit to animal data, these process models give trial-by-trial estimates of the animal’s changing associations as it learns. At the same time, new computational tools are emerging that make it easier than ever to fit complex models to large, hierarchical data sets. However, before behavior analysts can fully embrace this new set of tools, they will need to reconcile with its divergent terminology. On the one hand, the scope of ‘reinforcement’ in behavior analysis has broadened to include more properties of the reinforcer. Meanwhile, the term’s usage in computer science (and thus increasingly in neuroscience) is much less theoretically constrained than has been traditional in behavior analysis. It is now time to take stock of the new tools, their successes and limitations, and what role behavior analysis now has to play. Here, we sketch a reinforcement learning model of matching behavior, and use it to show how these new computational tools can help our search for the mechanisms of behavior.