⚙
AI-generated deep summary
by claude@2026-07, 2026-07-03
· read from full text
ⓘ
The paper studies how to model “fictive learning” in reinforcement learning, aiming to explain animal behavior in the rodent two-step task where agents switch between model-free and model-based control. The authors propose a brain-inspired learning rule based on a generalized reward prediction error that updates not only experienced outcomes but also unencountered states and actions, with the fictive update scaled by event correlations inferred from an internal model. In simulations and model fitting, the framework reproduces behavioral features that standard RL approaches (like Q-learning variants) fail to capture, and it also replicates dopaminergic dynamics observed in the same task; the main limitation is that validation is performed within the two-step task setting. The paper does not explicitly discuss endometriosis or adenomyosis; it was included in the corpus via a keyword match in the upstream search index.
Abstract
Reinforcement learning (RL) is a normative computational framework to account for reward-based learning. However, widely used RL models, including Q-learning and its variants, fail to capture some key behavioral phenomena observed in animal experiments, particularly the dynamic switch between model-free and model-based control in the two-step task. A fundamental discrepancy is that learning is restricted to experienced outcomes in the models, whereas biological agents may generalize learning to unvisited options based on internal world models, so-called fictive learning. We propose a simple, brain-inspired fictive learning rule and conduct the rodent two-step task to examine whether fictive learning could explain the observed behavior. The learning rule uses a generalized reward prediction error to update both experienced and non-encountered states and actions. The factual prediction error is scaled by the event correlation inferred from the internal model for fictive update. The generalized reward prediction error might be supported by brain-wide dopaminergic broadcasting. Through simulations, we show that this model reproduces key behavioral traits in the two-step task, including stay probabilities and regression analysis, which common RL models fail to explain. Model fitting validates its superior fit over existing alternatives. Furthermore, the model replicates dopaminergic dynamics observed in the same task. This framework bridges normative RL theory and biological learning, offering new insights into adaptive behavior.
Full text
1,637 characters
· extracted from
oa-doi-fallback
· click to expand
Abstract
Reinforcement learning (RL) is a normative computational framework to account for reward-based learning. However, widely used RL models, including Q-learning and its variants, fail to capture some key behavioral phenomena observed in animal experiments, particularly the dynamic switch between model-free and model-based control in the two-step task. A fundamental discrepancy is that learning is restricted to experienced outcomes in the models, whereas biological agents may generalize learning to unvisited options based on internal world models, so-called fictive learning. We propose a simple, brain-inspired fictive learning rule and conduct the rodent two-step task to examine whether fictive learning could explain the observed behavior. The learning rule uses a generalized reward prediction error to update both experienced and non-encountered states and actions. The factual prediction error is scaled by the event correlation inferred from the internal model for fictive update. The generalized reward prediction error might be supported by brain-wide dopaminergic broadcasting. Through simulations, we show that this model reproduces key behavioral traits in the two-step task, including stay probabilities and regression analysis, which common RL models fail to explain. Model fitting validates its superior fit over existing alternatives. Furthermore, the model replicates dopaminergic dynamics observed in the same task. This framework bridges normative RL theory and biological learning, offering new insights into adaptive behavior.
Competing Interest Statement
The authors have declared no competing interest.
Text is read by the "Ask this paper" AI Q&A widget below.
Extraction quality varies by source — PMC NXML preserves structure
cleanly, OA-HTML may include some navigation residue, and OA-PDF can
have broken hyphenation. The publisher copy
(via DOI)
is the canonical version.