Computational determinants of context-dependent preferences in probabilistic tasks.

A robust feature of human (and other animals) reinforcement learning is the influence of contextual factors on individual preferences. The behavioral hallmark of this effect is that when tested outside their original setting, options that were "best among bad alternatives" are often preferred over those that were "worst among good alternatives", regardless of their true objective value. Several computational processes have been proposed to explain the origin of such relative value encoding, most notably outcome normalization mechanisms such as reference-point dependence and range adaptation (primary relative encoding). However, it has also been suggested that context-dependent behavioral signatures may emerge spuriously from other computational processes, even when values are encoded in an absolute and unbiased manner (apparent relative encoding). Among these alternative accounts, choice perseveration and asymmetric update can be argued to play a role. To resolve this ambiguity, we employed a standard reinforcement learning protocol consisting of an initial learning phase, featuring rich and poor reward environments, followed by a transfer phase to assess cross-contextual preferences. Behavioral analyses of the transfer phase revealed robust signatures of context dependence across four independent datasets. We compared four distinct classes of computational models embodying primary relative encoding and apparent relative-value processes. Model simulations established the necessity of a hybrid architecture combining a primary relative value encoding mechanism with asymmetric update in order to adequately capture human behavior. While we developed a novel range-adaptation framework suitable for probabilistic tasks, Bayesian model comparison robustly favored reference-point dependence as the mechanism best adapted to the present contingencies. Overall, this work demonstrates that multiple computational biases coexist in human reinforcement learning and that signatures of context dependence cannot be dismissed as mere artifacts.