Adaptive Coding is Optimal in Reinforcement Learning

Adaptive coding is defined as the adjustment of parameters of a cognitive process as a function of the task's statistical properties. Our aim is to demonstrate that adaptive coding maximizes performance in situations where decision-making follows a reinforcement learning process (RLM), and test empirically test the predictions of our model. We first provide a detailed model of adaptive coding of reward in a RLM and study its performance, measured in expected value obtained from chosen options. In our model, the parameters modulating learning adjust optimally according to the information on outcomes. Their values are modulated by the trade-off between exploration and exploitation. When feedback is complete, exploration is redundant, and parameters adjust to maximize exploitation. When feedback is partial (only provided on the chosen option) an optimal compromise is reached between exploration and exploitation. We test these predictions in a set of experiments with complete and partial feedback. We find support for the hypothesis that coding is adaptive, and has the linear form we hypothesize. The design allows us to discriminate between two possible formulations, finding support for the hypothesis that adaptation is specific to reward encoding. We also find that the representational bias introduced in choice by the adaptive coding is transitory and is corrected when appropriate information is provided to participants.