Modeling Decanalization with Homeostatic Reinforcement Learning

Living organisms continuously manage their homeostatic setpoints but sometimes fall into the ruts and grooves of an entrenched or canalized behavioral repertoire. This manifests in a wide variety of ways, from overconsumption of scarce resources to repetitive unhealthy behavior, in cases of depression or anxiety. Clinicians are increasingly employing interventions, from psychoactive drugs to targeted stimulus regimes, to transiently decanalize entrenched behaviors and recanalize them into more adaptive patterns. These metaphors are long-standing, but have not been formally studied in explicit computational models. We present the first such model to our knowledge, a computational model of canalization, decanalization, and recanalization in a homeostatic navigation task. We compared homeostatic reinforcement learning (HRL) with a negative feedback (NF) reward function under matched canalization and decanalization conditions. In this setting, HRL agents showed improved post-decanalization homeostatic performance, relative to matched no-decanalization controls, whereas NF agents did not. We offer the model as a formal instantiation of a candidate mechanism by which temporary destabilization can produce more optimal behaviors in a constrained environment.