Expected reward value and reward prediction errors reinforce but also interfere with human time perception

Expected reward value and reward prediction errors reinforce but also interfere with human time perception | Research Square window.SnipcartSettings = { analytics: { enabled: false } }; (function() { var accessVector = localStorage.getItem('access_vector') || ''; window.dataLayer = window.dataLayer || []; if (accessVector) { window.dataLayer.push({ user: { profile: { profileInfo: { snid: accessVector } } } }); } })(); (function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src='https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);})(window,document,'script','dataLayer','GTM-K279D39R'); Browse Preprints In Review Journals COVID-19 Preprints AJE Video Bytes Research Tools Research Promotion AJE Professional Editing AJE Rubriq About Preprint Platform In Review Editorial Policies Our Team Advisory Board Help Center Sign In Submit a Preprint Cite Share Download PDF Article Expected reward value and reward prediction errors reinforce but also interfere with human time perception Emily K. DiMarco, Ashley Ratcliffe Shipp, Kenneth T. Kishida This is a preprint; it has not been peer reviewed by a journal. https://doi.org/ 10.21203/rs.3.rs-6968461/v1 This work is licensed under a CC BY 4.0 License Status: Under Revision Version 1 posted 14 You are reading this latest preprint version Abstract Time perception is often investigated using instrumental paradigms where reinforcement learning and associated dopaminergic processes have modulatory effects. Interval timing, which includes the judgment of relatively short intervals of time (ranging from milliseconds to minutes), has been shown to be modulated by manipulations of striatal dopamine. In reinforcement learning theory, the ‘expected value of reward’ (EV) and ‘reward prediction errors’ (RPEs) are key variables that explain striatal dopaminergic signals of reward processing during instrumental learning. Despite potential common dopaminergic underpinnings, the underlying connection between reinforcement learning processes and interval timing remains relatively underexplored. Herein, we investigated the impact of EV and RPEs on the human reproduction of 1000ms, 3000ms, and 5000ms intervals of time. Our results demonstrate that RPEs – specifically about rewards and not punishments – appear to reinforce performance errors, which effectively interfere with the rate at which reinforced 1000ms intervals – but not 3000ms and 5000ms intervals – are learned. The results of these experiments help clarify the role reinforcements play in interval timing, as well as give insight into the hypothetical mechanisms underlying time perception and the potential shared relationship with reinforcement learning processes. Biological sciences/Neuroscience/Cognitive neuroscience Biological sciences/Neuroscience/Cognitive neuroscience/Decision Biological sciences/Neuroscience/Cognitive neuroscience/Perception Figures Figure 1 Figure 2 Figure 3 Figure 4 Introduction The neurobiological mechanisms underlying the human perception of time and the execution of timing dependent behaviors remains elusive (Buhusi & Meck, 2005 ; Patton & Buonomano, 2018). Considerable evidence suggests that for relatively short time intervals, dopaminergic signals are critical (Meck, 1996 ; Petter, et al., 2018 ). To date, the best account of what dopamine neurons encode – and what dopamine release signals – lies with computational reinforcement learning theory (Sutton & Barto, 1998 ; Montague, et al., 1996 ; Shultz, et al., 1997; Uchida, et al., 2022; Sands, et al., 2024). Within this theoretical framework, ‘temporal difference reinforcement learning’ (TDRL), explains dynamic changes in dopamine neuron activity and changes in dopamine release throughout the course of learning Pavlovian and Instrumental associations (Montague, et al., 1996 ; Shultz, et al., 1997; Kim, et al., 2020; Sands, et al., 2023). An explanation of dopamine’s role in timing behavior ought to be possible in the context of TDRL’s theoretical terms; however, little work has been done to explicitly investigate TDRL defined variables in the context of time perception (though see Gershman, et al., 2014 ; Petter, et al., 2018 ; Mikhael & Gershman, 2019 ; and Jakob, et al., 2022 ). Here, we investigated interval timing and temporal learning while varying reward and punishment reinforcement and analyzed our data with a particular aim to disentangle TDRL terms and the corresponding hypothesized role for dopamine in time perception. Many reports have linked dopaminergic processes with time perception about relatively short intervals of time - on the order of hundreds of milliseconds to several seconds – known as interval timing (Buhusi & Meck, 2005 ). These behaviors are typically modelled in humans and rodents using interval timing tasks like peak interval procedures (Fig. 1 , Rakitin et al., 1998 ; Balci & Freestone, 2020). In humans, these tasks can be verbally instructed with little dependence on de novo learning required, though improvement in performance (i.e., temporal learning) can be observed over repeated trials. In rodents, however, as well as any other non-human animal model, interval timing must first be trained using instrumental reinforcement learning methods that utilize rewards to reinforce desired behaviors. Work attempting to use animal models to disentangle the role dopamine plays in the mechanisms underlying interval timing face the confound that dopaminergic processes and reward-based learning is required to train and elicit timing behaviors. Humans, on the other hand, can be instructed and are able to perform tasks without motivational drive provided by extrinsic rewards, which permits the comparison of reinforced versus non-reinforced timing behaviors and the impact that learning signals have on temporal learning and time perception. Through a variety of positive or negative reinforcement schedules, a wide range of Pavlovian and Instrumental learning effects can be trained and investigated in animal models and humans (MacInnis & Guilhardi, 2006 ). Amongst the most widely utilized approaches are those that use positive reinforcement via delivery of rewards to reinforce certain behaviors or environmental states. Both passive (i.e., Pavlovian) and active (i.e., Instrumental) learning have been shown to involve dopamine signaling; and, under both psychological constructs, time intervals were shown to be learned with accuracy and precision (Church, 2012 ). It has also been shown that dopamine neurons in non-human primate models respond in a manner consistent with a theorized optimal teaching signal called a “temporal difference reward prediction error” during Pavlovian conditioning (Montague, et al., 1996 ; Shultz, et al., 1997). Notably, dopamine neuron responses show precise time encoding as evidenced by the ‘pause’ in firing activity (indicating a negative reward prediction error) when – after learning – a reward was expected at a particular time but not delivered (Montague, et al., 1996 ; Shultz, et al., 1997). This result, suggesting that dopamine neurons and dopamine release encode a “temporal difference reward prediction error” during instrumental learning behavior has been replicated several times (Montague, et al., 2004 ; Glimcher, 2011 ; Kishida & Sands, 2021 ) in rodents (Uchida, et al., 2022) and in humans (Sands, et al., 2023). Dopamine’s role in time perception has been implicated by several complementary approaches in animal models and in humans. Neuropharmacological methods in human studies have shown that dopamine depletion (Coull et al., 2012 ), dopamine receptor antagonists (Drew et al., 2003 ; Buhusi & Meck, 2002 ) and psychostimulants that affect the dopamine system (Buhusi & Meck, 2002 ) alter time perception (Coull et al., 2012 ; Drew et al., 2003 ; Buhusi & Meck, 2002 ; Rammsayer, 1999 ). More recently, it has been demonstrated that rapid changes in dopamine neuron activity in mice (Jakob et al., 2022 ), manipulation of dopamine neuron activity using optogenetic methods in mice (Soares et al., 2016), and rapid (i.e., sub-second) and slow (i.e., several seconds) changes in dopamine levels in humans affect time perception (Sadibolova, et al., 2024; Sadibolova, et al., 2022 ; Terhune, et al., 2016 ). Notably, many of the methods used to implicate, measure, or manipulate dopamine levels during time perception are also used to investigate dopamine’s role in instrumental (and Pavlovian) reinforcement and learning (Frank & Fossella, 2011 ; Kishida & Sands, 2021 ). This has led us and others (Petter, et al., 2018 ; Mikhael & Gershman, 2019 ) to hypothesize that dopaminergic timing behavior may be mediated by the same mechanisms that underlie dopaminergic reinforcement learning. In this study, we explored the role reinforcement learning signals (i.e., ‘expected value of reward’ (EV) and reward prediction errors (RPEs)) play in interval timing in humans. Recently, these behavioral signals were shown to elicit a phasic (sub-second) change in dopamine levels in the human striatum during an instrumental learning task that did not modulate time intervals but did possess an explicit temporal structure (Sands, et al., 2023). We note that, according to the temporal difference RL framework, cues that predict a reward also elicit an EV-driven RPE, which is not described by the solely extrinsic-reward driven RPE described Rescorla-Wagner framework (Rescorla & Wagner, 1972 ). Thus, we hypothesized that changes in the EV of ‘criterion duration’ cues, associated RPEs, and RPEs following execution of timing behaviors with extrinsic reward or punishment would each modulate interval timing in a manner consistent with expected changes in striatal dopamine levels. To test our hypotheses, we executed two versions of a peak interval procedure (PIP) that investigated the reproduction of 1000ms, 3000ms, and 5000ms intervals of time (Fig. 1 A, 2 A). In the first task (Fig. 1 A), 24 participants ( Table 1 ) completed ‘PIP-neutral’ (PIP-n), which tested the reproduction of demonstrated criterion durations in the absence of any reinforcement. Next, 24 naïve participants ( Table 1 ) completed ‘PIP-reinforced’ (PIP-r, Fig. 2 A), which provided positive, negative, or no monetary reinforcement that scaled linearly with the accuracy of reproduction of the criterion durations (Fig. 2 B). We compared timing behavior in the PIP-neutral task to timing behavior in the PIP-reinforced task. Specifically, we determined and compared the rate of temporal learning when extrinsic (i.e., monetary) reinforcement was provided (PIP-r) compared to when it was absent (PIP-n). We also assessed the impact of RPEs on the performance of the same criterion durations in the subsequent trial. We report significant effects of the EV on accelerating temporal learning (Fig. 3 ). However, we also demonstrate that reinforcement can paradoxically interfere with temporal learning (Fig. 4 ). We discuss our results considering ‘temporal difference RL-theory’ and discuss how our results and this theoretical framework suggest the hypothesis that the brain can use dopaminergic reinforcement to learn ‘time’ intervals. Results Temporal learning without reinforcement. To characterize interval timing in response to predictable cues in the absence of reinforcement we designed PIP-neutral (PIP-n), which presented participants with nine different cues, three of each were predictive of 1000ms, 3000ms, and 5000ms, respectively ( Fig 1A ). Note, the triplicate design is intended to control for the number of cues presented in experiment 2 (PIP-r, described below). We recruited 24 participants ( Table 1 ) to complete this task and estimated temporal learning using fitted learning curves. Learning curves were fit for each participant based on absolute error of reproduced duration (please refer to methods, Eq. 1, for calculation of absolute error). Mean absolute error at each appearance was then fit with an exponential function see (methods, Eq. 2 ) to determine the rate of change in error overtime (i.e., temporal learning) ( Fig 1B ). The slope of the curve was used to estimate the rate of change in error over repeated trials. The intercept of the curve was used to estimate the initial error associated with each cue type ( Fig 1C ). Learning curves for each cued interval on the PIP-n showed significant temporal learning for all criterion durations ( Fig 1B ). Specifically, participants showed significantly lower absolute error at the last appearance (appearance 16) when compared to the first appearance (appearance one) for all cue types ( Fig 1B ; two-way mixed ANOVA; F(1,46) = 10.910; p = 2.00e-03**; generalized eta squared (ges) = 0.081, indicates medium effect size). Mean intercepts of learning curves for each criterion duration also show larger absolute error for longer criterion durations, as reported in previous works (Rammsayer, & Troche, 2014) ( Fig 1C ; one-way mixed ANOVA; F(2,69) = 12.589; p = 2.18e-05***; ges = 0.267, indicates large effect size). As expected, we did not find significant differences in learning rates or intercepts between neutral cues on the PIP-n within the same criterion duration tested ( Fig S1 ). Also, in accordance with other reproduction tasks (Rammsayer, et al., 2015), on average, we observed a general underestimation of all intervals tested ( Fig S1 ). Table 1. Participant demographics and task performance. PIP-n (n = 24) PIP-r (n = 24) Mean age (years) 32.92 ± 12.75 34.67 ± 14.35 Gender Male: 10 Female: 13 Other: 1 Male: 7 Female: 16 Other: 1 Handedness Right: 17 Left: 6 Ambidextrous: 1 Right: 21 Left: 0 Ambidextrous: 3 Task time (minutes) 22.86 ± 0.57 22.13 ± 0.52 Total points earned 107.4 ± 15.34 92.45 ± 15.25** Data expressed as Mean ± SD. PIP = Peak Interval Procedure. Total points equates to monetary outcome for each task. **Indicates that there is a statistically significant difference in total points between PIP-n and PIP-r participants. Actual participant compensation was scaled based on performance and total study time. Learning curves for each cued interval on the PIP-n showed significant temporal learning for all criterion durations ( Fig 1B ). Specifically, participants showed significantly lower absolute error at the last appearance (appearance 16) when compared to the first appearance (appearance one) for all cue types ( Fig 1B ; two-way mixed ANOVA; F(1,46) = 10.910; p = 2.00e-03**; generalized eta squared (ges) = 0.081, indicates medium effect size). Mean intercepts of learning curves for each criterion duration also show larger absolute error for longer criterion durations, as reported in previous works (Rammsayer, & Troche, 2014) ( Fig 1C ; one-way mixed ANOVA; F(2,69) = 12.589; p = 2.18e-05***; ges = 0.267, indicates large effect size). As expected, we did not find significant differences in learning rates or intercepts between neutral cues on the PIP-n within the same criterion duration tested ( Fig S1 ). Also, in accordance with other reproduction tasks (Rammsayer, et al., 2015), on average, we observed a general underestimation of all intervals tested ( Fig S1 ). Temporal learning with positive and negative reinforcement. To test the hypothesis that positive and negative reinforcement of temporal reproduction would increase temporal learning we designed and executed the PIP-reinforced (PIP-r, Fig 2A ). During PIP-r, a reinforcement was paired with a specific temporal cue presented for 1000, 3000 or 5000ms. Reinforcers were a reward (monetary gain) or punishment (monetary loss) delivered immediately following the participant’s reproduced duration. The amount of monetary reinforcement was calculated linearly from the absolute value of the temporal error of the participant on that trial ( Fig 2B ; Eq 1 ; see methods for a description of reinforcement dispersion) scaled to a maximum of ±$3. Three cues indicated a reward, one for each the 1000ms, 3000ms, and 5000ms criterion durations. Another three cues indicated a punishment, one for each the 1000ms, 3000ms, and 5000ms criterion durations. Lastly, three additional cues indicated no reinforcement, again one for each the 1000ms, 3000ms, and 5000ms criterion durations. To ensure there were no effects of learning priors on performance, we recruited 24 naive participants to complete the PIP-r and estimated their interval timing using fitted learning curves ( Fig 2C ) as described for experiment 1 (PIP-n). We first compared the baseline performance between participants on the PIP-n and PIP-r using the total accumulated points on the task, and we found that participants earned significantly more points on the PIP-n compared to the PIP-r ( Table 1 ; two-way t-test; t(46) = 3.3879, p-value = 0.0015**). Consistent with behavior on the PIP-n, participants show significantly lower error at the last appearance (appearance 16) when compared to the first appearance for all cues on the PIP-r ( Fig 2C ; two-way mixed ANOVA; F(1,46) = 11.756; p = 0.001**; ges = 0.090). We hypothesized that there would be increased rates of temporal learning in response to the reinforcement schedule on the PIP-r, compared to the lack of reward schedule on the PIP-n. We compared the slope and intercepts of the learning curves from the PIP-n versus PIP-r ( Fig 3 and S3 ). We found significantly higher intercepts ( Fig 3A ; mixed ANOVA; F(1,94) = 20.096; p = 2.08e-05***; ges = 0.176), but significantly steeper slopes ( Fig 3B ; mixed ANOVA; F(1,94) = 9.421; p = 0.003**; ges = 0.091) for the neutral cue on PIP-r when compared to the neutral cues on the PIP-n for the 1000ms duration. Notably, there were no significant differences between the neutral cues on the PIP-n and PIP-r for the 3000ms ( Fig S3B; mixed ANOVA; F(1,94) = 0.359; p = 0.551; ges = 0.004; Fig S3E ; mixed ANOVA; F(1,94) = 1.972; p = 0.164; ges = 0.021) and 5000ms ( Fig S3C; F(1,94) = 1.349; p = 0.248; ges = 0.014; Fig S3F ; mixed ANOVA; F(1,94) = 3.656; p = 0.059; ges = 0.037) durations. Thus, participants exhibited a specific increase in learning rates in response to reinforced interval timing tasks (PIP-r) for the 1000ms criterion duration but not for the 3000 or 5000ms ms durations. Reward Prediction Errors (RPEs) for rewards, but not punishments, reinforce performance errors. Based on our observation that introducing a reinforcement schedule can increase the initial error (higher intercept of learning curve) but also accelerates temporal learning (increased slope of learning curve) for neutral 1000ms trials and the extant literature, we hypothesized that reinforcement learning mechanisms involving reward prediction errors to actual outcomes and changes in expectations about rewards and punishments drive accelerated learning. Therefore, we calculated RPEs based on the expected value (EV) value of a particular cue (i.e., estimated as their average earnings up to that point on the task for that cue type) and what was actually received on that trial (i.e., the monetary reward or punishment based on accuracy on that cue, see Eq. 3 in methods). We then fit linear regression models to determine if RPEs on the previous trial are associated with performance prediction errors on the current trial. Performance prediction errors were calculated based on the difference between the expected reproduced duration on a given cue and the actual reproduced duration on the cue ( Eq 4 ; described in methods). From the comparison of RPEs on the previous trial with performance prediction errors on the current trial, we found a significant positive correlation between RPEs on the previous trial and performance prediction errors on the current trial for positively reinforced cues on the PIP-r ( Fig 4, Experiment 2 ; linear regression model; F(1,343) = 14.79; p = 0.0001***; R 2 adj = 0.0386, indicates large effect size). Interestingly, however, we did not see a relationship between RPEs and PPEs on negatively reinforced cues on the PIP-r ( Fig 4, Experiment 2; linear regression model; F(1,360) = 0.2985; p = 0.5851; R 2 adj = -0.002). We also calculated ‘fictive-RPEs’ for neutral cues based on what participants would have accrued if the cues had been reinforced. We compared previous trial ‘fictive-RPEs’ on the with actual PPEs on the current trial, and we found a significant association for the neutral cues on the PIP-r ( Fig 4, Experiment 2 ; linear regression model; F(1,350) = 4.461; p = 0.0354*; R 2 adj = 0.0097). In experiment 1, no reinforcers were provided, yet temporal learning occurred. To determine if the effect of RPE on performance in experiment 2 was specific to reinforcement and not an artifact of general performance improvements we again estimated ‘fictive-RPEs’ based on what participants would have earned if the cues had reinforced and compared the relationship between fictive-RPEs and performance prediction errors during PIP-n. In PIP-r, the RPE is strongly associated with decreases in performance error over time and is a signal that participants observe. On the other hand, the ‘fictive-RPE’ is not observed by participants and should only be associated with the PPE in PIP-n if the effect observed in PIP-r was driven by general performance improvement and not an explicit effect on the actual rewards accrued. We observed no significant associations between fictive-RPEs and performance prediction errors for any cues in PIP-n ( Fig 4, Experiment 1 ; linear regression models; Neutral-1: F(1,335) = 3.103; p = 0.07906; R 2 adj = 0.0062; Neutral-2: F(1,343) = 1.513; p = 0.2195; R 2 adj = 0.0015; Neutral-3: F(1,339) = 0.0567; p = 0.8119; R 2 adj = -0.0028). To ensure that the described differences in the timing between the PIP-n and PIP-r groups were due to the instilled task conditions and not due to differences in basic timing behavior or speed of learning between groups, we performed a third experiment on 40 naive participants with the same task design as the PIP-r but with different magnitude of reinforcement. We found consistent results as performance showed similar increases in temporal learning rates during 1000ms positively reinforced cues that scaled to the magnitude of reinforcer (see supplemental results; Fig S5A-B ). We also found consistent effects of a significant positive correlation between RPEs on the previous trial and performance prediction errors on the current trial for positively reinforced cues ( see supplemental results; Fig S5, Experiment 3 ). Discussion The current study investigated temporal learning and interval timing in the presence and absence of reinforcement. Over two experiments, a total of 48 individuals performed a version of a peak interval procedure (PIP) reproducing time intervals of 1000ms, 3000ms, and 5000ms (Fig. 1 and Fig. 2 ). The expectation of positive or negative reinforcement was associated with significantly higher initial error (Fig. 3 A) but also was associated with accelerated temporal learning (Fig. 3 B). These effects were specific for 1000ms intervals and were not observed for 3000ms and 5000ms. Notably, a closer examination of the role of a potent reinforcement signal, the reward prediction error (RPE), revealed that trial-by-trial errors, and not performance improvements, were reinforced by ‘actual win’ associated RPEs but not ‘actual loss’ associated RPEs. The effect of ‘actual win’ related RPEs appeared to generalize to the neutral cues (or non-reinforced cues in PIP-r). In model-free temporal difference reinforcement learning theory, cues that acquire the expectation of wins (or losses) are sufficient to drive dopaminergic reward prediction errors. Our results are consistent with the hypothesis that phasic dopamine responses drive expectations associated with cues on the PIP-r, but also to actual wins on the PIP-r driving paradoxical performance changes over the course of the PIP-r. It is possible that early, uncertain expectations increase the salience of cues and drive an initial increase in error in the reproduction of durations. Over repeated trials, temporal learning naturally occurs to overcome this error but appears to be interfered with by RPEs that reinforce ‘bad’ behavior (Fig. 4 , Experiment 2). Notably, the fastest learning in PIP-r occurs for the non-reinforced neutral cues (Fig. 3 B) where the expectation of an explicit reward (monetary gain) or punishment (monetary loss) is absent. To understand the effects of reinforcers on interval timing during an instrumental conditioning task, we first measured interval timing on a PIP in the absence of reinforcement (Fig. 1 A). We showed that participants exhibited temporal learning of 1000ms, 3000ms, and 5000ms intervals of time, as evidenced by the decreased absolute error at the last appearance of the cue compared to the first appearance (Fig. 1 B). We then sought to characterize how reinforcers would affect this temporal learning, so we designed a second experiment to test temporal learning on a PIP in the presence of positive reinforcers, negative reinforcers, and non-reinforced (neutral) cues (Fig. 2 A). We expected temporal learning rates to increase for positive and negative reinforcers, as participants could adjust their behaviors from monetary feedback. However, within PIP-r, we saw no differences in learning rates, reproduced durations, or accuracy between reinforced and non-reinforced cues (Fig. 2 B). We hypothesized that this result may be due to the generalization of behavior from reinforced cues to non-reinforced cues on this task. Therefore, we compared temporal learning rates from the PIP in the absence of reinforcement from our first experiment (Fig. 1 ) to temporal learning on the PIP in presence of reinforcement from our second experiment (Fig. 2 ). We found that the presence of reinforcement on an interval timing task increased temporal learning rates when compared to a task without reinforcement (Fig. 3 ). Interestingly, this increase in temporal learning rate was due to an increase in initial temporal error experienced on the reinforced version of the PIP when compared to the non-reinforced version of the PIP. To ensure these timing effects weren’t due to overall group level differences or additional cognitive load due to the introduction of reinforcement, we performed an supplementary experiment in 40 naïve participants that demonstrated that these interval timing effects were consistent in the face of reinforcers, and they scaled to the magnitude of reinforcement expected ( Fig S5A ; supplemental results). As the structure of the PIP used to measure interval timing in this study is similar to instrumental conditioning paradigms, it is possible that temporal learning is being accelerated through RPE-mediated mechanisms. Prior to this study, the role of reinforcers as a teaching signal, specifically for interval timing, was unclear. Therefore, we investigated whether RPEs associated with interval timing could help explain differences in temporal learning and temporal errors generated in the context of different valence of reinforcement. Our results show that RPEs on previous trials are associated with temporal performance prediction errors (PPEs) on the current trial for positively reinforced 1000ms cues (Fig. 4 ), which suggests that RPEs are reinforcing the poor performance exhibited on the previous trial. Previous research investigating the role of dopamine in interval timing have led some to propose that reinforcement learning models could be used to explain the dopamine response to interval timing (Petter, et al., 2018 ; Mikhael & Gershman 2019 ). As delaying or emitting a reinforcer has been shown to cause the silencing of the dopamine response at the expected time of reward, it has been hypothesized that the dopamine response may track temporal errors as part of or in addition to errors in reward valuation (Hollerman & Schultz 1998 ). RPE-driven phasic dopamine signals at the outset of each trial and at the end of each trial may be one possible explanation of this increased temporal error but also increased learning rate in the context of a task with reinforcement. Our study provides evidence that the presence of reinforcement increases the rate of temporal learning for intervals of time of 1000ms in duration ( Fig S3A ), but not 3000ms or 5000ms durations ( Fig S3B-C ). This result is aligned with other research, as different interval timing behavioral effects have been described when comparing durations of less than and greater than approximately 1200ms, with individual differences in the breakpoint (Artieda, et al., 1992 ; Koch, et al., 2008). It has been hypothesized that different dopaminergic circuitry may be involved in the perception of durations of time less than and greater than approximately 1200ms (Hinton & Meck, 2004 ). While dopaminergic action in the striatum is thought to play a role in the perception of time of intervals greater than approximately 350ms, the prefrontal cortex and other “decision-making centers” are believed to be engaged when perceiving intervals greater than about 1200ms, as perceiving longer intervals of time is thought to require working memory and temporal planning processes (Smith & Jonides, 1999 ; Hinton & Meck, 2004 ; Meck 2005 ). Our results provide additional evidence of a behavioral difference between timing intervals 1000ms in duration and intervals of greater than 3000ms in duration, as well as may provide some insight into the potential underlying dopaminergic mechanism for intervals of a 1000ms in duration. Our results demonstrate a significant effect of learning signals that are shared between instrumental learning paradigms and methods used to investigate interval timing in animal models. In humans, the peak-interval procedure that we used to study interval timing can be instructed and participant performance can be measured, allowing for initial temporal learning effects to be observed. In the TDRL framework, there are two key variables that contribute to the calculation of a reward prediction error. We investigate the impact these signals have on temporal learning by comparing temporal learning behavior when these signals are present (PIP-r) versus absent (PIP-n). In TDRL, cues that predict a future possible reward can generate a reward prediction error in the absence of a concurrent reward. It was recently shown that TD-RPEs can elicit a phasic increase in dopamine levels in human striatum (Sands et al., 2023 ). Together this suggests that the RPE effects observed here may be due to hypothesized phasic increases in dopamine. Interestingly, Sands et al., also showed that dopamine levels rose to the absence of punishment when a punishment was expected (a punishment prediction error) but fell when the punishment was worse than expected. In the current results, the RPE on expected punishment trials failed to reinforce poor performance but also did not significantly enhance temporal learning. In the context of rewards and punishments, cues that predict neither (the neutral cues in PIP-r) led to significantly faster temporal learning than neutral cues in PIP-n. One possible explanation would be that the neutral cues in the context of PIP-r gained an expected value that was non-punishing and therefore surprisingly good (better than expected relative to a possibly punishing trial) even though they were not reinforced positively. We did not model the TDRL-RPE in the current work due to significant challenges to modeling the peak-interval procedures without making significant and currently unresolvable assumptions about how the state-space of the task may be represented in the participants’ brains. Together, our results suggest a hypothetical role of reinforcement learning mediated dopamine signals in time perception. Others have suggested such a role (Gershman, et al., 2014 ; Petter, et al., 2018 ; Mikhael & Gershman, 2019 ; and Jakob, et al., 2022 ) but have proposed models that deviate from the simplest and most well-established model for dopaminergic neuron activity and dopamine release (i.e., TDRL representations). Future work should be aimed at understanding whether the expected value and reward prediction error effects observed here in humans is indeed mediated by associated changes in dopamine in human striatum in line with the TDRL framework. Nevertheless, we have demonstrated that error signals, previously demonstrated to modulate phasic dopamine levels, can modulate timing behavior and temporal learning in paradoxical ways. Without directly measuring dopamine during these behaviors in the absence of confounding behavioral training, it will be difficult to tease out the role dopamine plays specifically in timing behaviors and temporal learning versus effects that are likely associated with instrumental learning paradigms requires to have non-human animals perform these behaviors. While dopamine, reward prediction errors, and timing behaviors are clearly interconnected, significant work remains to develop a formal model of the time we perceive. Materials and Methods Experimental Design. We recruited and consented two groups of 24 participants each (48 total) from the Winston-Salem, North Carolina region using methods approved by the Wake Forest University School of Medicine IRB (IRB00042265) for a one-hour study visit ( Table 1 ). All research was performed in accordance with relevant guidelines/regulations per Wake Forest University School of Medicine IRB, and informed consent was obtained from all participants in accordance with the Declaration of Helsinki. One participant was excluded from the study due to accidental data loss. Participants sat approximately 2 feet away from a Dell computer monitor and placed their dominant pointer finger on a button from a hand-held button box. Participants were then instructed on how to play the Peak Interval Procedure (PIP). The task was designed in Python 3 using the PyGame library. On initialization of the task, nine visual cues were randomly selected from a pool of 60 fractal images sourced from free online stock images. Each cue was assigned an interval of time (3 cues to each category: 1000ms, 3000ms, or 5000ms) (Fig. 1 A, Fig. 2 A). For the task that included reinforcements, cues were further divided into categories of reinforcement (win, loss, or nothing) (Fig. 2 A). The maximum reinforcement amount was scaled to $ 3 on all trials on the PIP-r. The reinforcement was calculated using a linear decay of maximum reinforcement in relation to the error in the reproduction time (for example, a reproduction of 950ms for a 1000ms win trial would equate to a reinforcement of + $ 0.95 whereas a reproduction of 950ms for a 1000ms loss trial will equate to a reinforcement of - $ 0.05). Cues were randomly presented by placing each cue into an array 16 times and shuffling to determine the sequence of trials for a total of 144 trials. Stages of the task included: presentation, prompt, reproduction, and reinforcement. During the presentation stage, participants observed the presented cue for the assigned duration (1000ms, 3000ms, or 5000ms) (Fig. 1 A, 2 A). Following a black 500ms screen, a ‘ready’ prompt was displayed for 1000ms. After another 500ms black screen, the cue reappeared. Participants were instructed to use the hand-held button box to reproduce the duration of time by pressing the button when the cued duration elapsed. The cue disappeared when the button was pressed or after twice the cued duration passed. Immediately after, a monetary reinforcement or a hairpin cross was displayed based on the reinforcement type. After reinforcement, or if the cue type was neutral, the trial was complete. The cue disappeared, and a black screen was shown for an interval of time randomly drawn from a Poisson distribution with lambda equal to 1500ms. On average, experiments lasted ~ 23 minutes per participant ( Table 1 ). At the end of the study visit, participants were compensated based on the study duration ( $ 5 per 15 minutes), with a bonus based on task performance (ranging from $ 20 - $ 60 additional). Analysis was completed using R Studio. Experiment 1. To characterize interval timing in response to predictable cues in the absence of reinforcement, we created the PIP-neutral (PIP-n), which presented participants with nine different cues, three of each were predictive of 1000ms, 3000ms, and 5000ms criterion durations, respectively (Fig. 1 A). We recruited 24 participants to complete this task and measured temporal learning using learning curves (Fig. 1 B ) . Learning curves were calculated for each participant based on absolute value of the temporal error for each cue type at each appearance, using Eq. (1): Eq 1 Temporal Accuracy = |RD-CD| Where RD is the participant’s reproduced duration and CD is the criterion duration. Mean absolute value of the error at each appearance was then fit with an exponential function to determine the rate of temporal learning overtime, using Eq. (2): Eq 2 Y i = B 0 + B 1 *log(X i ) Where Y is absolute error for each cue (i). X is each appearance of the cue (i). B 0 represents the intercept and measures the value where the line crosses the y-axis. B 1 is the slope of the curve and measures the rate of change in temporal error or the temporal learning rate. Experiment 2. We modified the PIP to deliver reinforcements on 2/3rds of all trials immediately following the participant’s button press, in a task called the PIP-reinforced (PIP-r; Fig. 2 A). During the PIP-r, a reinforcement was a reward (monetary gain) or punishment (monetary loss) immediately following the reproduced duration. The monetary reinforcement structure was the amount of money a participant gained or lost (scaled to ± $ 3) on a trial and was calculated in a linear fashion from the temporal accuracy of the participant on that trial (Fig. 2 B). Temporal accuracy was calculated using Eq. (1). Where RD was the participant’s reproduced duration and CD was the criterion duration (1000ms, 3000ms, or 5000ms). To ensure there were no effects of priors on performance, we recruited 24 naive participants to complete the PIP-r and measured their interval timing using learning curves (Fig. 2 C). Reward Prediction Errors. We calculated reward prediction errors as the difference between what participants might have expected to earn on a given cue and what they actually received (Fig. 4 ). Reward prediction error (RPE) was calculated using Eq. (3): Eq 3 RPE t = V t – \(\:\frac{1}{t-1}\sum\:_{1}^{t-1}{V}_{(t-1)}\) Where t is the current trial of the same cue type (criterion duration x reinforcement type) and V is the monetary value of that trial. Performance prediction errors (PPE) were calculated based on the difference between the expected reproduced duration on a given cue and how participants actually responded to the cue using Eq. (4): Eq 4 PPE t = RD t – \(\:\frac{1}{t-1}\sum\:_{1}^{t-1}{RD}_{(t-1)}\) Where t is the current trial with the same cue type (criterion duration x reinforcement type) and RD is the reproduced duration on that trial. Statistical Analysis. Sample size was determined by an a priori power analysis. A group-level statistical assessment of the relationship between mean differences in temporal accuracy from a previous study (DiMarco et al. 2023 ) while anticipating a large (0.8) effect size at power level of 0.8 and p-value of 0.01 for a two-tailed t-test, required recruiting 14–32 participants (per group). These estimates do not use sex or age as factors. Large outliers were removed from analysis. Statistics were performed using R Studio (Posit Team 2024 ). Mixed effects ANOVAs were used to compare cue types within and across tasks, accounting for repeated measures across the same subjects. Multiple pairwise t-tests with Bonferroni correction were used to subsequently determine significance between individual cue types while adjusting the significance level to account for increased type I error due to multiple comparisons. Generalized eta squared (ges) was used to determine effect size within each ANOVA comparison. Linear regression models were used to determine associations between RPEs and PPEs across reinforcement context. Declarations Acknowledgments We would like to extend our gratitude to our human participants for volunteering for this study and for their commitment to research. Funding: National Institutes of Health grant KL2TR00142 National Institutes of Health grant R01 DA048096 National Institutes of Health grant R01 MH121099 National Institutes of Health grant R01 NS092701 National Institutes of Health grant R01 MH1241 Author contributions: Conceptualization: EKD, KTK Methodology: EKD, ARS, KTK Investigation: EKD Visualization: EKD Supervision: KTK Writing—original draft: EKD, KTK Writing—review & editing: EKD, ARS, KTK Competing interests: Authors declare that they have no competing interests. Data and materials availability: All data are available in the main text or the supplementary materials. References Artieda, J., Pastor, M. A., Lacruz, F., & Obeso, J. A. (1992). Temporal discrimination is abnormal in Parkinson's disease. Brain: a journal of neurology, 115 Pt 1, 199–210. https://doi.org/10.1093/brain/115.1.199 Balcı, F., & Freestone, D. (2020). The Peak Interval Procedure in Rodents: A Tool for Studying the Neurobiological Basis of Interval Timing and Its Alterations in Models of Human Disease. Bio-protocol, 10(17), e3735. https://doi.org/10.21769/BioProtoc.3735 Buhusi, C. V., & Meck, W. H. (2002). Differential effects of methamphetamine and haloperidol on the control of an internal clock. Behavioral Neuroscience, 116 (2), 291–297. https://doi.org/10.1037/0735-7044.116.2.291 Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nature reviews. Neuroscience , 6 (10), 755–765. https://doi.org/10.1038/nrn1764 Church, R.M. (2012). Temporal Learning in Humans and Other Animals. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_66 Coull, J. T., Hwang, H. J., Leyton, M., & Dagher, A. (2012). Dopamine precursor depletion impairs timing in healthy volunteers by attenuating activity in putamen and supplementary motor area. The Journal of neuroscience: the official journal of the Society for Neuroscience, 32(47), 16704–16715. https://doi.org/10.1523/JNEUROSCI.1258-12.2012 DiMarco, E. K., Sadibolova, R., Jiang, A., Liebenow, B., Jones, R. E., Haq, I. U., Siddiqui, M. S., Terhune, D. B., & Kishida, K. T. (2023). Time perception reflects individual differences in motor and non-motor symptoms of Parkinson's disease. Parkinsonism & related disorders , 114 , 105800. https://doi.org/10.1016/j.parkreldis.2023.105800 Drew, M. R., Fairhurst, S., Malapani, C., Horvitz, J. C., & Balsam, P. D. (2003). Effects of dopamine antagonists on the timing of two intervals. Pharmacology, biochemistry, and behavior, 75(1), 9–15. https://doi.org/10.1016/s0091-3057(03)00036-4 Frank, M. J., & Fossella, J. A. (2011). Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology, 36(1), 133–152. https://doi.org/10.1038/npp.2010.96 Gershman, S. J., Moustafa, A. A., & Ludvig, E. A. (2014). Time representation in reinforcement learning models of the basal ganglia. Frontiers in computational neuroscience, 7, 194. https://doi.org/10.3389/fncom.2013.00194 Glimcher P. W. (2011). Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences of the United States of America , 108 Suppl 3 (Suppl 3), 15647–15654. https://doi.org/10.1073/pnas.1014269108 Hinton, S. C., & Meck, W. H. (2004). Frontal-striatal circuitry activated by human peak-interval timing in the supra-seconds range. Brain research. Cognitive brain research , 21 (2), 171–182. https://doi.org/10.1016/j.cogbrainres.2004.08.005 Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature neuroscience, 1(4), 304–309. https://doi.org/10.1038/1124 Jakob, A. M. V., Mikhael, J. G., Hamilos, A. E., Assad, J. A., & Gershman, S. J. (2022). Dopamine mediates the bidirectional update of interval timing. Behavioral neuroscience , 136 (5), 445–452. https://doi.org/10.1037/bne0000529 Kim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui-Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe-Uchida, M., Gershman, S. J., & Uchida, N. (2020). A Unified Framework for Dopamine Signals across Timescales. Cell , 183 (6), 1600–1616.e25. https://doi.org/10.1016/j.cell.2020.11.013 Kishida K. T., & Sands L. P. (2021). A Dynamic Affective Core to Bind the Contents, Context, and Value of Conscious Experience. Affect Dynamics, 293-328:8. Koch, G., Costa, A., Brusa, L., Peppe, A., Gatto, I., Torriero, S., Gerfo, E. L., Salerno, S., Oliveri, M., Carlesimo, G. A., & Caltagirone, C. (2008). Impaired reproduction of second but not millisecond time intervals in Parkinson's disease. Neuropsychologia, 46(5), 1305–1313. https://doi.org/10.1016/j.neuropsychologia.2007.12.005 MacInnis, M.L.M., & Guilhardi, P. (2006). Basic interval discrimination procedures. In M. A. Anderson (Ed.), Tasks and Techniques: A Sampling of Methodologies for the Investigation of Animal Learning, Behavior, and Cognition, pp. 233-244. Hauppauge, NY: Nova Science Publishers. SBN 1-60021-126-7. Meck W. H. (1996). Neuropharmacology of timing and time perception. Brain research. Cognitive brain research, 3(3-4), 227–242. https://doi.org/10.1016/0926-6410(96)00009-2 Meck W. H. (2005). Neuropsychology of timing and time perception. Brain and cognition , 58 (1), 1–8. https://doi.org/10.1016/j.bandc.2004.09.004 Mikhael, J. G., & Gershman, S. J. (2019). Adapting the flow of time with dopamine. Journal of neurophysiology , 121 (5), 1748–1760. https://doi.org/10.1152/jn.00817.2018 Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature , 431 (7010), 760–767. https://doi.org/10.1038/nature03015 Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of neuroscience: the official journal of the Society for Neuroscience , 16 (5), 1936–1947. https://doi.org/10.1523/JNEUROSCI.16-05-01936.1996 Paton, J. J., & Buonomano, D. V. (2018). The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions. Neuron, 98(4), 687–705. https://doi.org/10.1016/j.neuron.2018.03.045 Petter, E. A., Gershman, S. J., & Meck, W. H. (2018). Integrating Models of Interval Timing and Reinforcement Learning. Trends in cognitive sciences , 22 (10), 911–922. https://doi.org/10.1016/j.tics.2018.08.004 Posit team (2024). RStudio: Integrated Development Environment for R. Posit Software, PBC, Boston, MA. URL http://www.posit.co/. Rakitin, B. C., Gibbon, J., Penney, T. B., Malapani, C., Hinton, S. C., & Meck, W. H. (1998). Scalar expectancy theory and peak-interval timing in humans. Journal of experimental psychology. Animal behavior processes, 24(1), 15–33. https://doi.org/10.1037//0097-7403.24.1.15 Rammsayer T. H. (1999). Neuropharmacological evidence for different timing mechanisms in humans. The Quarterly journal of experimental psychology. B, Comparative and physiological psychology, 52(3), 273–286. https://doi.org/10.1080/713932708 Rammsayer, T. H., & Troche, S. J. (2014). In search of the internal structure of the processes underlying interval timing in the sub-second and the second range: A confirmatory factor analysis approach. Acta Psychologica, 147, 68–74. https://doi.org/10.1016/j.actpsy.2013.05.004 Rammsayer, T. H., Borter, N., & Troche, S. J. (2015). Visual-auditory differences in duration discrimination of intervals in the subsecond and second range. Frontiers in psychology , 6 , 1626. https://doi.org/10.3389/fpsyg.2015.01626 Rescorla R. A., Wagner A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. Classical conditioning, Current research and theory , 2 , 64-69. Sadibolova, R., Monaldi, L., & Terhune, D. B. (2022). A proxy measure of striatal dopamine predicts individual differences in temporal precision. Psychonomic bulletin & review, 29(4), 1307–1316. https://doi.org/10.3758/s13423-022-02077-1 Sadibolova, R., DiMarco, E. K., Jiang, A., Maas, B., Tatter, S. B., Laxton, A., Kishida, K. T., & Terhune, D. B. (2024). Sub-second and multi-second dopamine dynamics underlie variability in human time perception. medRxix: the preprint server for health sciences , 2024.02.09.24302276. https://doi.org/10.1101/2024.02.09.24302276 Sands, L. P., Jiang, A., Liebenow, B., DiMarco, E., Laxton, A. W., Tatter, S. B., Montague, P. R., & Kishida, K. T. (2023). Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans. Science advances , 9 (48), eadi4927. https://doi.org/10.1126/sciadv.adi4927 Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science (New York, N.Y.) , 275 (5306), 1593–1599. https://doi.org/10.1126/science.275.5306.1593 Smith, E. E., & Jonides, J. (1999). Storage and executive processes in the frontal lobes. Science (New York, N.Y.) , 283 (5408), 1657–1661. https://doi.org/10.1126/science.283.5408.1657 Sutton R, Barto A. Reinforcement Learning: An Introduction. The MIT Press, First edition, (1998). Terhune, D. B., Sullivan, J. G., & Simola, J. M. (2016). Time dilates after spontaneous blinking. Current biology: CB , 26 (11), R459–R460. https://doi.org/10.1016/j.cub.2016.04.010 Additional Declarations No competing interests reported. Supplementary Files scientificReportssupplementarymaterials.pdf Cite Share Download PDF Status: Under Revision Version 1 posted Editorial decision: Revision requested 11 Feb, 2026 Reviews received at journal 09 Feb, 2026 Reviews received at journal 09 Feb, 2026 Reviews received at journal 09 Feb, 2026 Reviewers agreed at journal 01 Feb, 2026 Reviewers agreed at journal 01 Feb, 2026 Reviewers agreed at journal 31 Jan, 2026 Reviews received at journal 07 Oct, 2025 Reviewers agreed at journal 24 Sep, 2025 Reviewers invited by journal 22 Jul, 2025 Editor assigned by journal 22 Jul, 2025 Editor invited by journal 26 Jun, 2025 Submission checks completed at journal 25 Jun, 2025 First submitted to journal 24 Jun, 2025 You are reading this latest preprint version Research Square lets you share your work early, gain feedback from the community, and start making changes to your manuscript prior to peer review in a journal. As a division of Research Square Company, we’re committed to making research communication faster, fairer, and more useful. We do this by developing innovative software and high quality services for the global research community. Our growing team is made up of researchers and industry professionals working together to solve the most critical problems facing scientific publishing. Also discoverable on Platform About Our Team In Review Editorial Policies Advisory Board Help Center Resources Author Services Accessibility API Access RSS feed Manage Cookie Preferences © Research Square 2026 | ISSN 2693-5015 (online) Privacy Policy Terms of Service Do Not Sell My Personal Information {"props":{"pageProps":{"initialData":{"identity":"rs-6968461","acceptedTermsAndConditions":true,"allowDirectSubmit":false,"archivedVersions":[],"articleType":"Article","associatedPublications":[],"authors":[{"id":489966562,"identity":"b1657a4a-9c8e-4daf-9245-db49833751af","order_by":0,"name":"Emily K. DiMarco","email":"","orcid":"","institution":"Wake Forest University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Emily","middleName":"K.","lastName":"DiMarco","suffix":""},{"id":489966563,"identity":"bd90fe5a-1f37-4506-aecd-ecfc006a10b1","order_by":1,"name":"Ashley Ratcliffe Shipp","email":"","orcid":"","institution":"Wake Forest University School of Medicine","correspondingAuthor":false,"prefix":"","firstName":"Ashley","middleName":"Ratcliffe","lastName":"Shipp","suffix":""},{"id":489966564,"identity":"932c1b38-18bb-497d-b654-c6859e91cc48","order_by":2,"name":"Kenneth T. Kishida","email":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAZAAAAAyAQMAAABI0h/eAAAABlBMVEX///8AAABVwtN+AAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAuElEQVRIiWNgGAWjYBACCWYGNoYEtgQ5BuYDQC4bCVqMGdgSiNUCVsaWkNhAtBbJdvZnDx6UpaVvOMZjwPCh7DBhLdLMPOYGCedyckFaGGecI0KLHDMPm0RiW0Xutvs9Bsy8bURpYX8G0pJuBrSF+S8xWqSZGcyAWnISwFoYidEi2cxjJpFwLs1w/zG2goM959IJa5E4f/yZ5I+yZHnJNuaND36UWRPWggIOkKh+FIyCUTAKRgEuAABCPTVIdxpDawAAAABJRU5ErkJggg==","orcid":"","institution":"Wake Forest University School of Medicine","correspondingAuthor":true,"prefix":"","firstName":"Kenneth","middleName":"T.","lastName":"Kishida","suffix":""}],"badges":[],"createdAt":"2025-06-24 19:08:12","currentVersionCode":1,"declarations":"","doi":"10.21203/rs.3.rs-6968461/v1","doiUrl":"https://doi.org/10.21203/rs.3.rs-6968461/v1","draftVersion":[],"editorialEvents":[],"editorialNote":"","failedWorkflow":false,"files":[{"id":87521516,"identity":"836518eb-e601-4716-8234-7ffcf37385c9","added_by":"auto","created_at":"2025-07-24 18:00:32","extension":"jpg","order_by":1,"title":"Figure 1","display":"","copyAsset":false,"role":"figure","size":68846,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSignificant temporal learning occurs in response to non-reinforced cues. A. \u003c/strong\u003eA trial of the PIP-n (PIP-neutral) began with a ’Cue Onset’, where a cue was presented on the screen for a criterion duration (CD; 5000ms, 3000ms, or 1000ms), indicated by distinct cues. Following the presentation of a CD, a black screen appeared for 500ms followed by a ‘ready’ prompt (fixed 1000ms) to prepare the participant to reproduce the cued CD. Following another 500ms black screen, the cue was then re-presented and remained on the screen until the participant reproduced the cued CD with a button press, or a time twice the CD passed. \u003cstrong\u003eB.\u003c/strong\u003e Learning curves were calculated for each participant (n = 24) based on absolute error for each cue type at each appearance. Mean absolute error at each appearance was then fit with an exponential function to determine the rate of temporal learning overtime. Learning curves for the representative neutral-2 cues on the PIP-n show significantly lower error at appearance 16 when compared to appearance one. \u003cstrong\u003eC. \u003c/strong\u003eWhen comparing intercepts of learning curves across criterion durations, intercepts increased as CDs increased, revealing increasing error for larger intervals. \u003cem\u003eSignificance based on p \u0026lt; 0.05*, p \u0026lt; 0.01**, p \u0026lt; 0.001***. Shading = SEM.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"1.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/e62daf4bc8a455182db39135.jpg"},{"id":87521517,"identity":"c36b87fc-617a-4ba6-8ac6-31c4215e5607","added_by":"auto","created_at":"2025-07-24 18:00:32","extension":"jpg","order_by":2,"title":"Figure 2","display":"","copyAsset":false,"role":"figure","size":88905,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eSignificant temporal learning occurs in response to reinforced cues. A.\u003c/strong\u003e A trial of the PIP-r (PIP-reinforced) began with a ’Cue Onset’, where a cue was presented on the screen for a criterion duration (CD; 5000ms, 3000ms, or 1000ms), indicated by distinct cues. Following the presentation of a CD, a black screen appeared for 500ms followed by a ‘ready’ prompt (fixed 1000ms) to prepare the participant to reproduce the cued CD. Following another 500ms black screen, the cue was then re-presented and remained on the screen until the participant reproduced the cued CD with a button press, or a time twice the CD passed. Reinforcements (win or loss) were presented following the button press (fixed 500ms). \u003cstrong\u003eB. \u003c/strong\u003eReinforcements were linearly scaled to the absolute difference between the reproduced duration (RD) and the CD, with a maximum possible reward or loss of ±$3 \u003cstrong\u003eC.\u003c/strong\u003e Learning curves were calculated for each participant (n = 24) based on absolute error for each cue type at each appearance. Mean absolute error at each appearance was then fit with an exponential function to determine the rate of temporal learning overtime. Learning curves for each cue type on PIP-r show significantly lower error at the last appearance (16) when compared to the first appearance (1), for all cues. \u003cem\u003eSignificance based on p \u0026lt; 0.05*.Shading = SEM.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"2.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/e43df41963da67a506ae9db9.jpg"},{"id":87522270,"identity":"b898f490-800c-4424-8305-2990965d2de6","added_by":"auto","created_at":"2025-07-24 18:08:32","extension":"jpg","order_by":3,"title":"Figure 3","display":"","copyAsset":false,"role":"figure","size":54628,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eReinforcements increase the intercepts of the learning curves, which generalizes to the neutral cues on 1000ms duration. A. \u003c/strong\u003eThe intercepts of the learning curves are significantly different between the neutral cues (Neutral-1, Neutral-2, Neutral-3) on the PIP-n and all cues (Neutral-r, Win-r, Loss-r) on the PIP-r for the 1000ms criterion duration. \u003cem\u003e“!” denotes group is significantly different than all three neutral groups (blue) based on pair-wise t-tests. \u003c/em\u003e\u0026nbsp;\u003cstrong\u003eB. \u003c/strong\u003eThe slopes of the learning curves are significantly different between the neutral cues on the PIP-n and the neutral cue on the PIP-r for the 1000ms CD. \u003cem\u003e“#” denotes group is significantly different than all three neutral groups (blue) based on pair-wise t-tests. Significance based on one-way mixed measures ANOVAs followed by post-hoc pairwise t-tests.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"3.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/359819745e012444b499c8a4.jpg"},{"id":87521518,"identity":"ce6dfcd0-2e4e-437d-9f1a-34974ad47ff7","added_by":"auto","created_at":"2025-07-24 18:00:32","extension":"jpg","order_by":4,"title":"Figure 4","display":"","copyAsset":false,"role":"figure","size":119569,"visible":true,"origin":"","legend":"\u003cp\u003e\u003cstrong\u003eReward prediction errors (RPEs) on previous trials (n-1) are associated with performance prediction errors (PPEs) on the current trial (n) for positively reinforced cues. Experiment 1.\u003c/strong\u003e Linear regression models show the results of the association between reward prediction errors of the previous trial (n-1) of the same type and performance prediction errors on the current trail (n) for the 1000ms cues on the PIP-n. \u003cstrong\u003eExperiment 2. \u003c/strong\u003efor the cues presented on the PIP-r. \u003cem\u003eSignificance based on p \u0026lt; 0.05*, 0.01**, 0.001***. Shading = SEM.\u003c/em\u003e\u003c/p\u003e","description":"","filename":"4.jpg","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/b2e6f4012a24f465e89d0704.jpg"},{"id":87523323,"identity":"96a04fe1-945d-431b-b8f8-2a7ee35c55eb","added_by":"auto","created_at":"2025-07-24 18:32:33","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"manuscript-pdf","size":1325985,"visible":true,"origin":"","legend":"","description":"","filename":"manuscript.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/4003e412-66e6-4ebc-898d-89387c8d3c4a.pdf"},{"id":87521522,"identity":"318ae035-dbbf-4527-a0fe-b7678c05d84d","added_by":"auto","created_at":"2025-07-24 18:00:32","extension":"pdf","order_by":0,"title":"","display":"","copyAsset":false,"role":"supplement","size":2051387,"visible":true,"origin":"","legend":"","description":"","filename":"scientificReportssupplementarymaterials.pdf","url":"https://assets-eu.researchsquare.com/files/rs-6968461/v1/250781744998923337773ce7.pdf"}],"financialInterests":"No competing interests reported.","formattedTitle":"Expected reward value and reward prediction errors reinforce but also interfere with human time perception","fulltext":[{"header":"Introduction","content":"\u003cp\u003eThe neurobiological mechanisms underlying the human perception of time and the execution of timing dependent behaviors remains elusive (Buhusi \u0026amp; Meck, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2005\u003c/span\u003e; Patton \u0026amp; Buonomano, 2018). Considerable evidence suggests that for relatively short time intervals, dopaminergic signals are critical (Meck, \u003cspan citationid=\"CR19\" class=\"CitationRef\"\u003e1996\u003c/span\u003e; Petter, et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2018\u003c/span\u003e). To date, the best account of what dopamine neurons encode \u0026ndash; and what dopamine release signals \u0026ndash; lies with computational reinforcement learning theory (Sutton \u0026amp; Barto, \u003cspan citationid=\"CR37\" class=\"CitationRef\"\u003e1998\u003c/span\u003e; Montague, et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e1996\u003c/span\u003e; Shultz, et al., 1997; Uchida, et al., 2022; Sands, et al., 2024). Within this theoretical framework, \u0026lsquo;temporal difference reinforcement learning\u0026rsquo; (TDRL), explains dynamic changes in dopamine neuron activity and changes in dopamine release throughout the course of learning Pavlovian and Instrumental associations (Montague, et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e1996\u003c/span\u003e; Shultz, et al., 1997; Kim, et al., 2020; Sands, et al., 2023). An explanation of dopamine\u0026rsquo;s role in timing behavior ought to be possible in the context of TDRL\u0026rsquo;s theoretical terms; however, little work has been done to explicitly investigate TDRL defined variables in the context of time perception (though see Gershman, et al., \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Petter, et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Mikhael \u0026amp; Gershman, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; and Jakob, et al., \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2022\u003c/span\u003e). Here, we investigated interval timing and temporal learning while varying reward and punishment reinforcement and analyzed our data with a particular aim to disentangle TDRL terms and the corresponding hypothesized role for dopamine in time perception.\u003c/p\u003e\u003cp\u003eMany reports have linked dopaminergic processes with time perception about relatively short intervals of time - on the order of hundreds of milliseconds to several seconds \u0026ndash; known as interval timing (Buhusi \u0026amp; Meck, \u003cspan citationid=\"CR4\" class=\"CitationRef\"\u003e2005\u003c/span\u003e). These behaviors are typically modelled in humans and rodents using interval timing tasks like peak interval procedures (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003e, Rakitin et al., \u003cspan citationid=\"CR27\" class=\"CitationRef\"\u003e1998\u003c/span\u003e; Balci \u0026amp; Freestone, 2020). In humans, these tasks can be verbally instructed with little dependence on \u003cem\u003ede novo\u003c/em\u003e learning required, though improvement in performance (i.e., temporal learning) can be observed over repeated trials. In rodents, however, as well as any other non-human animal model, interval timing must first be trained using instrumental reinforcement learning methods that utilize rewards to reinforce desired behaviors. Work attempting to use animal models to disentangle the role dopamine plays in the mechanisms underlying interval timing face the confound that dopaminergic processes and reward-based learning is required to train and elicit timing behaviors. Humans, on the other hand, can be instructed and are able to perform tasks without motivational drive provided by extrinsic rewards, which permits the comparison of reinforced versus non-reinforced timing behaviors and the impact that learning signals have on temporal learning and time perception.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003eThrough a variety of positive or negative reinforcement schedules, a wide range of Pavlovian and Instrumental learning effects can be trained and investigated in animal models and humans (MacInnis \u0026amp; Guilhardi, \u003cspan citationid=\"CR18\" class=\"CitationRef\"\u003e2006\u003c/span\u003e). Amongst the most widely utilized approaches are those that use positive reinforcement via delivery of rewards to reinforce certain behaviors or environmental states. Both passive (i.e., Pavlovian) and active (i.e., Instrumental) learning have been shown to involve dopamine signaling; and, under both psychological constructs, time intervals were shown to be learned with accuracy and precision (Church, \u003cspan citationid=\"CR5\" class=\"CitationRef\"\u003e2012\u003c/span\u003e). It has also been shown that dopamine neurons in non-human primate models respond in a manner consistent with a theorized optimal teaching signal called a \u0026ldquo;temporal difference reward prediction error\u0026rdquo; during Pavlovian conditioning (Montague, et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e1996\u003c/span\u003e; Shultz, et al., 1997). Notably, dopamine neuron responses show precise time encoding as evidenced by the \u0026lsquo;pause\u0026rsquo; in firing activity (indicating a negative reward prediction error) when \u0026ndash; after learning \u0026ndash; a reward was expected at a particular time but not delivered (Montague, et al., \u003cspan citationid=\"CR23\" class=\"CitationRef\"\u003e1996\u003c/span\u003e; Shultz, et al., 1997). This result, suggesting that dopamine neurons and dopamine release encode a \u0026ldquo;temporal difference reward prediction error\u0026rdquo; during instrumental learning behavior has been replicated several times (Montague, et al., \u003cspan citationid=\"CR22\" class=\"CitationRef\"\u003e2004\u003c/span\u003e; Glimcher, \u003cspan citationid=\"CR11\" class=\"CitationRef\"\u003e2011\u003c/span\u003e; Kishida \u0026amp; Sands, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2021\u003c/span\u003e) in rodents (Uchida, et al., 2022) and in humans (Sands, et al., 2023).\u003c/p\u003e\u003cp\u003eDopamine\u0026rsquo;s role in time perception has been implicated by several complementary approaches in animal models and in humans. Neuropharmacological methods in human studies have shown that dopamine depletion (Coull et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2012\u003c/span\u003e), dopamine receptor antagonists (Drew et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2003\u003c/span\u003e; Buhusi \u0026amp; Meck, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2002\u003c/span\u003e) and psychostimulants that affect the dopamine system (Buhusi \u0026amp; Meck, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2002\u003c/span\u003e) alter time perception (Coull et al., \u003cspan citationid=\"CR6\" class=\"CitationRef\"\u003e2012\u003c/span\u003e; Drew et al., \u003cspan citationid=\"CR8\" class=\"CitationRef\"\u003e2003\u003c/span\u003e; Buhusi \u0026amp; Meck, \u003cspan citationid=\"CR3\" class=\"CitationRef\"\u003e2002\u003c/span\u003e; Rammsayer, \u003cspan citationid=\"CR28\" class=\"CitationRef\"\u003e1999\u003c/span\u003e). More recently, it has been demonstrated that rapid changes in dopamine neuron activity in mice (Jakob et al., \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2022\u003c/span\u003e), manipulation of dopamine neuron activity using optogenetic methods in mice (Soares et al., 2016), and rapid (i.e., sub-second) and slow (i.e., several seconds) changes in dopamine levels in humans affect time perception (Sadibolova, et al., 2024; Sadibolova, et al., \u003cspan citationid=\"CR32\" class=\"CitationRef\"\u003e2022\u003c/span\u003e; Terhune, et al., \u003cspan citationid=\"CR38\" class=\"CitationRef\"\u003e2016\u003c/span\u003e). Notably, many of the methods used to implicate, measure, or manipulate dopamine levels during time perception are also used to investigate dopamine\u0026rsquo;s role in instrumental (and Pavlovian) reinforcement and learning (Frank \u0026amp; Fossella, \u003cspan citationid=\"CR9\" class=\"CitationRef\"\u003e2011\u003c/span\u003e; Kishida \u0026amp; Sands, \u003cspan citationid=\"CR16\" class=\"CitationRef\"\u003e2021\u003c/span\u003e). This has led us and others (Petter, et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Mikhael \u0026amp; Gershman, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2019\u003c/span\u003e) to hypothesize that dopaminergic timing behavior may be mediated by the same mechanisms that underlie dopaminergic reinforcement learning.\u003c/p\u003e\u003cp\u003eIn this study, we explored the role reinforcement learning signals (i.e., \u0026lsquo;expected value of reward\u0026rsquo; (EV) and reward prediction errors (RPEs)) play in interval timing in humans. Recently, these behavioral signals were shown to elicit a phasic (sub-second) change in dopamine levels in the human striatum during an instrumental learning task that did not modulate time intervals but did possess an explicit temporal structure (Sands, et al., 2023). We note that, according to the temporal difference RL framework, cues that predict a reward also elicit an EV-driven RPE, which is not described by the solely extrinsic-reward driven RPE described Rescorla-Wagner framework (Rescorla \u0026amp; Wagner, \u003cspan citationid=\"CR31\" class=\"CitationRef\"\u003e1972\u003c/span\u003e). Thus, we hypothesized that changes in the EV of \u0026lsquo;criterion duration\u0026rsquo; cues, associated RPEs, and RPEs following execution of timing behaviors with extrinsic reward or punishment would each modulate interval timing in a manner consistent with expected changes in striatal dopamine levels.\u003c/p\u003e\u003cp\u003eTo test our hypotheses, we executed two versions of a peak interval procedure (PIP) that investigated the reproduction of 1000ms, 3000ms, and 5000ms intervals of time (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA, \u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). In the first task (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA), 24 participants (\u003cb\u003eTable\u0026nbsp;1\u003c/b\u003e) completed \u0026lsquo;PIP-neutral\u0026rsquo; (PIP-n), which tested the reproduction of demonstrated criterion durations in the absence of any reinforcement. Next, 24 na\u0026iuml;ve participants (\u003cb\u003eTable\u0026nbsp;1\u003c/b\u003e) completed \u0026lsquo;PIP-reinforced\u0026rsquo; (PIP-r, Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA), which provided positive, negative, or no monetary reinforcement that scaled linearly with the accuracy of reproduction of the criterion durations (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eB). We compared timing behavior in the PIP-neutral task to timing behavior in the PIP-reinforced task. Specifically, we determined and compared the rate of temporal learning when extrinsic (i.e., monetary) reinforcement was provided (PIP-r) compared to when it was absent (PIP-n). We also assessed the impact of RPEs on the performance of the same criterion durations in the subsequent trial. We report significant effects of the EV on accelerating temporal learning (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e3\u003c/span\u003e). However, we also demonstrate that reinforcement can paradoxically interfere with temporal learning (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e). We discuss our results considering \u0026lsquo;temporal difference RL-theory\u0026rsquo; and discuss how our results and this theoretical framework suggest the hypothesis that the brain can use dopaminergic reinforcement to learn \u0026lsquo;time\u0026rsquo; intervals.\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e\u003cp\u003e\u003c/p\u003e"},{"header":"Results","content":"\u003cp\u003e\u003cstrong\u003eTemporal learning without reinforcement.\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo characterize interval timing in response to predictable cues in the absence of reinforcement we designed PIP-neutral (PIP-n), which presented participants with nine different cues, three of each were predictive of 1000ms, 3000ms, and 5000ms, respectively (\u003cstrong\u003eFig 1A\u003c/strong\u003e). Note, the triplicate design is intended to control for the number of cues presented in experiment 2 (PIP-r, described below). We recruited 24 participants (\u003cstrong\u003eTable 1\u003c/strong\u003e) to complete this task and estimated temporal learning using fitted learning curves. Learning curves were fit for each participant based on absolute error of reproduced duration (please refer to methods, \u003cem\u003eEq. 1,\u0026nbsp;\u003c/em\u003efor calculation of absolute error). Mean absolute error at each appearance was then fit with an exponential function see (methods, \u003cem\u003eEq. 2\u003c/em\u003e) to determine the rate of change in error overtime (i.e., temporal learning) (\u003cstrong\u003eFig 1B\u003c/strong\u003e). \u0026nbsp;The slope of the curve was used to estimate the rate of change in error over repeated trials. The intercept of the curve was used to estimate the initial error associated with each cue type (\u003cstrong\u003eFig 1C\u003c/strong\u003e).\u003c/p\u003e\n\u003cp\u003eLearning curves for each cued interval on the PIP-n showed significant temporal learning for all criterion durations (\u003cstrong\u003eFig 1B\u003c/strong\u003e). Specifically, participants showed significantly lower absolute error at the last appearance (appearance 16) when compared to the first appearance (appearance one) for all cue types (\u003cstrong\u003eFig 1B\u003c/strong\u003e; two-way mixed ANOVA; F(1,46) = 10.910; p = 2.00e-03**; generalized eta squared (ges) = 0.081, indicates medium effect size). Mean intercepts of learning curves for each criterion duration also show larger absolute error for longer criterion durations, as reported in previous works (Rammsayer, \u0026amp; Troche, 2014) (\u003cstrong\u003eFig 1C\u003c/strong\u003e; one-way mixed ANOVA; F(2,69) = 12.589; p = 2.18e-05***; ges = 0.267, indicates large effect size). As expected, we did not find significant differences in learning rates or intercepts between neutral cues on the PIP-n within the same criterion duration tested (\u003cstrong\u003eFig S1\u003c/strong\u003e). Also, in accordance with other reproduction tasks (Rammsayer, et al., 2015), on average, we observed a general underestimation of all intervals tested (\u003cstrong\u003eFig S1\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTable 1. Participant demographics and task performance.\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003ctable border=\"0\" cellspacing=\"0\" cellpadding=\"0\" width=\"421\"\u003e\n \u003ctbody\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\u003cbr\u003e\u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePIP-n (n = 24)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e\u003cstrong\u003ePIP-r (n = 24)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eMean age (years)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e32.92\u0026nbsp;\u0026plusmn;\u0026nbsp;12.75\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e34.67\u0026nbsp;\u0026plusmn;\u0026nbsp;14.35\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eGender\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003eMale: 10\u003c/p\u003e\n \u003cp\u003eFemale: 13\u003c/p\u003e\n \u003cp\u003eOther: 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003eMale: 7\u003c/p\u003e\n \u003cp\u003eFemale: 16\u003c/p\u003e\n \u003cp\u003eOther: 1\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eHandedness\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003eRight: 17\u003c/p\u003e\n \u003cp\u003eLeft: 6\u003c/p\u003e\n \u003cp\u003eAmbidextrous: 1\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003eRight: 21\u003c/p\u003e\n \u003cp\u003eLeft: 0\u003c/p\u003e\n \u003cp\u003eAmbidextrous: 3\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTask time (minutes)\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e22.86 \u0026plusmn; 0.57\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e22.13 \u0026plusmn; 0.52\u0026nbsp;\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003ctr\u003e\n \u003ctd style=\"width: 162px;\"\u003e\n \u003cp\u003e\u003cstrong\u003eTotal points earned\u003c/strong\u003e\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e107.4 \u0026plusmn; 15.34\u003c/p\u003e\n \u003c/td\u003e\n \u003ctd style=\"width: 129px;\"\u003e\n \u003cp\u003e92.45 \u0026plusmn; 15.25**\u003c/p\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n \u003c/tbody\u003e\n\u003c/table\u003e\n\u003cp\u003eData expressed as Mean \u0026plusmn; SD. \u0026nbsp;PIP = Peak Interval Procedure. Total points equates to monetary outcome for each task. **Indicates that there is a statistically significant difference in total points between PIP-n and PIP-r participants. Actual participant compensation was scaled based on performance and total study time.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eLearning curves for each cued interval on the PIP-n showed significant temporal learning for all criterion durations (\u003cstrong\u003eFig 1B\u003c/strong\u003e). Specifically, participants showed significantly lower absolute error at the last appearance (appearance 16) when compared to the first appearance (appearance one) for all cue types (\u003cstrong\u003eFig 1B\u003c/strong\u003e; two-way mixed ANOVA; F(1,46) = 10.910; p = 2.00e-03**; generalized eta squared (ges) = 0.081, indicates medium effect size). Mean intercepts of learning curves for each criterion duration also show larger absolute error for longer criterion durations, as reported in previous works (Rammsayer, \u0026amp; Troche, 2014) (\u003cstrong\u003eFig 1C\u003c/strong\u003e; one-way mixed ANOVA; F(2,69) = 12.589; p = 2.18e-05***; ges = 0.267, indicates large effect size). As expected, we did not find significant differences in learning rates or intercepts between neutral cues on the PIP-n within the same criterion duration tested (\u003cstrong\u003eFig S1\u003c/strong\u003e). Also, in accordance with other reproduction tasks (Rammsayer, et al., 2015), on average, we observed a general underestimation of all intervals tested (\u003cstrong\u003eFig S1\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eTemporal learning with positive and negative reinforcement.\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eTo test the hypothesis that positive and negative reinforcement of temporal reproduction would increase temporal learning we designed and executed the\u0026nbsp;PIP-reinforced (PIP-r, \u003cstrong\u003eFig 2A\u003c/strong\u003e). During PIP-r, a reinforcement was paired with a specific temporal cue presented for 1000, 3000 or 5000ms. Reinforcers were a reward (monetary gain) or punishment (monetary loss) delivered immediately following the participant\u0026rsquo;s reproduced duration. The amount of monetary reinforcement was calculated linearly from the absolute value of the temporal error of the participant on that trial (\u003cstrong\u003eFig 2B\u003c/strong\u003e; \u003cem\u003eEq 1\u003c/em\u003e;\u0026nbsp;see methods for a description of reinforcement dispersion) scaled to a maximum of \u0026plusmn;$3. \u0026nbsp;Three cues indicated a reward, one for each the 1000ms, 3000ms, and 5000ms criterion durations. Another three cues indicated a punishment, one for each the 1000ms, 3000ms, and 5000ms criterion durations. \u0026nbsp; Lastly, three additional cues indicated no reinforcement, again one for each the 1000ms, 3000ms, and 5000ms criterion durations. To ensure there were no effects of learning priors on performance, we recruited 24 naive participants to complete the PIP-r and estimated their interval timing using fitted learning curves (\u003cstrong\u003eFig 2C\u003c/strong\u003e) as described for experiment 1 (PIP-n).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eWe first compared the baseline performance between participants on the PIP-n and PIP-r using the total accumulated points on the task, and we found that participants earned significantly more points on the PIP-n compared to the PIP-r (\u003cstrong\u003eTable 1\u003c/strong\u003e;\u003cstrong\u003e\u0026nbsp;\u003c/strong\u003etwo-way t-test; t(46) = 3.3879, p-value = 0.0015**). Consistent with behavior on the PIP-n, participants show significantly lower error at the last appearance (appearance 16) when compared to the first appearance for all cues on the PIP-r (\u003cstrong\u003eFig 2C\u003c/strong\u003e; two-way mixed ANOVA; F(1,46) = 11.756; p = 0.001**; ges = 0.090). We hypothesized that there would be increased rates of temporal learning in response to the reinforcement schedule on the PIP-r, compared to the lack of reward schedule on the PIP-n. We compared the slope and intercepts of the learning curves from the PIP-n versus PIP-r (\u003cstrong\u003eFig 3 and S3\u003c/strong\u003e). We found significantly higher intercepts (\u003cstrong\u003eFig 3A\u003c/strong\u003e; mixed ANOVA; F(1,94) = 20.096; p = 2.08e-05***; ges = 0.176), but significantly steeper slopes (\u003cstrong\u003eFig 3B\u003c/strong\u003e; mixed ANOVA; F(1,94) = 9.421; p = 0.003**; ges = 0.091) for the neutral cue on PIP-r when compared to the neutral cues on the PIP-n for the 1000ms duration.\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eNotably, there were no significant differences between the neutral cues on the PIP-n and PIP-r for the 3000ms (\u003cstrong\u003eFig S3B;\u0026nbsp;\u003c/strong\u003emixed ANOVA; F(1,94) = 0.359; p = 0.551; ges = 0.004;\u0026nbsp;\u003cstrong\u003eFig S3E\u003c/strong\u003e; mixed ANOVA; F(1,94) = \u0026nbsp;1.972; p = 0.164; ges = 0.021) and 5000ms (\u003cstrong\u003eFig S3C;\u0026nbsp;\u003c/strong\u003eF(1,94) = 1.349; p = \u0026nbsp;0.248; ges = 0.014;\u0026nbsp;\u003cstrong\u003eFig S3F\u003c/strong\u003e; mixed ANOVA; F(1,94) = 3.656; p = 0.059; ges = 0.037) durations. Thus, participants exhibited a specific increase in learning rates in response to\u0026nbsp;reinforced interval timing tasks (PIP-r) for the 1000ms criterion duration but not for the 3000 or 5000ms ms durations.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eReward Prediction Errors (RPEs) for rewards, but not punishments, reinforce performance errors.\u0026nbsp;\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eBased on our observation that introducing a reinforcement schedule can increase the initial error (higher intercept of learning curve) but also accelerates temporal learning (increased slope of learning curve) for neutral 1000ms trials and the extant literature, we hypothesized that reinforcement learning mechanisms involving reward prediction errors to actual outcomes and changes in expectations about rewards and punishments drive accelerated learning. Therefore, we calculated RPEs based on the expected value (EV) value of a particular cue (i.e., estimated as their average earnings up to that point on the task for that cue type) and what was actually received on that trial (i.e., the monetary reward or punishment based on accuracy on that cue, see \u003cem\u003eEq. 3\u003c/em\u003e in methods).\u0026nbsp;We then fit linear regression models to determine if RPEs on the previous trial are associated with performance prediction errors on the current trial. Performance prediction errors were calculated\u0026nbsp;based on the difference between the expected reproduced duration on a given cue and the actual reproduced duration on the cue (\u003cem\u003eEq 4\u003c/em\u003e;\u0026nbsp;described in methods).\u003c/p\u003e\n\u003cp\u003eFrom the comparison of RPEs on the previous trial with performance prediction errors on the current trial, we found a significant positive correlation between RPEs on the previous trial and performance prediction errors on the current trial for positively reinforced cues on the PIP-r (\u003cstrong\u003eFig 4, Experiment 2\u003c/strong\u003e; linear regression model; F(1,343) = 14.79; p = 0.0001***; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = 0.0386, indicates large effect size). Interestingly, however, we did not see a relationship between RPEs and PPEs on negatively reinforced cues on the PIP-r (\u003cstrong\u003eFig 4, Experiment 2;\u0026nbsp;\u003c/strong\u003elinear regression model; F(1,360) = 0.2985; p = 0.5851; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = -0.002). \u0026nbsp;We also calculated \u0026lsquo;fictive-RPEs\u0026rsquo; for neutral cues based on what participants would have accrued if the cues had been reinforced. We compared previous trial \u0026lsquo;fictive-RPEs\u0026rsquo; on the with actual PPEs on the current trial, and we found a significant association for the neutral cues on the PIP-r (\u003cstrong\u003eFig 4, Experiment 2\u003c/strong\u003e; linear regression model; F(1,350) = 4.461; p = 0.0354*; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = 0.0097).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eIn experiment 1, no reinforcers were provided, yet temporal learning occurred. To determine if the effect of RPE on performance in experiment 2 was specific to reinforcement and not an artifact of general performance improvements we again estimated \u0026lsquo;fictive-RPEs\u0026rsquo; based on what participants would have earned if the cues had reinforced and compared the relationship between fictive-RPEs and performance prediction errors during PIP-n. In PIP-r, the RPE is strongly associated with decreases in performance error over time and is a signal that participants observe. On the other hand, the \u0026lsquo;fictive-RPE\u0026rsquo; is not observed by participants and should only be associated with the PPE in PIP-n if the effect observed in PIP-r was driven by general performance improvement and not an explicit effect on the actual rewards accrued. We observed no significant associations between fictive-RPEs and performance prediction errors for any cues in PIP-n (\u003cstrong\u003eFig 4, Experiment 1\u003c/strong\u003e; linear regression models; Neutral-1: F(1,335) = 3.103; p = 0.07906; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = 0.0062; Neutral-2: F(1,343) = 1.513; p = 0.2195; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = 0.0015; Neutral-3: F(1,339) = 0.0567; p = 0.8119; R\u003csup\u003e2\u003c/sup\u003e\u003csub\u003eadj\u003c/sub\u003e = -0.0028).\u0026nbsp;\u003c/p\u003e\n\u003cp\u003eTo ensure that the described differences in the timing between the PIP-n and PIP-r groups were due to the instilled task conditions and not due to differences in basic timing behavior or speed of learning between groups, we performed a third experiment on 40 naive participants with the same task design as the PIP-r but with different magnitude of reinforcement. \u0026nbsp;We found consistent results as performance showed similar increases in temporal learning rates during 1000ms positively reinforced cues that scaled to the magnitude of reinforcer (see supplemental results;\u003cstrong\u003e\u0026nbsp;Fig S5A-B\u003c/strong\u003e). \u0026nbsp;We also found consistent effects of a significant positive correlation between RPEs on the previous trial and performance prediction errors on the current trial for positively reinforced cues (\u003cstrong\u003esee\u0026nbsp;\u003c/strong\u003esupplemental results;\u003cstrong\u003e\u0026nbsp;Fig S5, Experiment 3\u003c/strong\u003e).\u0026nbsp;\u003c/p\u003e"},{"header":"Discussion","content":"\u003cp\u003eThe current study investigated temporal learning and interval timing in the presence and absence of reinforcement. Over two experiments, a total of 48 individuals performed a version of a peak interval procedure (PIP) reproducing time intervals of 1000ms, 3000ms, and 5000ms (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003e \u003cb\u003eand\u003c/b\u003e Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003e). The expectation of positive or negative reinforcement was associated with significantly higher initial error (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e3\u003c/span\u003eA) but also was associated with accelerated temporal learning (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e3\u003c/span\u003eB). These effects were specific for 1000ms intervals and were not observed for 3000ms and 5000ms. Notably, a closer examination of the role of a potent reinforcement signal, the reward prediction error (RPE), revealed that trial-by-trial errors, and not performance improvements, were reinforced by \u0026lsquo;actual win\u0026rsquo; associated RPEs but not \u0026lsquo;actual loss\u0026rsquo; associated RPEs. The effect of \u0026lsquo;actual win\u0026rsquo; related RPEs appeared to generalize to the neutral cues (or non-reinforced cues in PIP-r). In model-free temporal difference reinforcement learning theory, cues that acquire the expectation of wins (or losses) are sufficient to drive dopaminergic reward prediction errors. Our results are consistent with the hypothesis that phasic dopamine responses drive expectations associated with cues on the PIP-r, but also to actual wins on the PIP-r driving paradoxical performance changes over the course of the PIP-r. It is possible that early, uncertain expectations increase the salience of cues and drive an initial increase in error in the reproduction of durations. Over repeated trials, temporal learning naturally occurs to overcome this error but appears to be interfered with by RPEs that reinforce \u0026lsquo;bad\u0026rsquo; behavior (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e, Experiment 2). Notably, the fastest learning in PIP-r occurs for the non-reinforced neutral cues (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e3\u003c/span\u003eB) where the expectation of an explicit reward (monetary gain) or punishment (monetary loss) is absent.\u003c/p\u003e\u003cp\u003eTo understand the effects of reinforcers on interval timing during an instrumental conditioning task, we first measured interval timing on a PIP in the absence of reinforcement (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA). We showed that participants exhibited temporal learning of 1000ms, 3000ms, and 5000ms intervals of time, as evidenced by the decreased absolute error at the last appearance of the cue compared to the first appearance (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eB). We then sought to characterize how reinforcers would affect this temporal learning, so we designed a second experiment to test temporal learning on a PIP in the presence of positive reinforcers, negative reinforcers, and non-reinforced (neutral) cues (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). We expected temporal learning rates to increase for positive and negative reinforcers, as participants could adjust their behaviors from monetary feedback. However, \u003cem\u003ewithin\u003c/em\u003e PIP-r, we saw no differences in learning rates, reproduced durations, or accuracy between reinforced and non-reinforced cues (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eB). We hypothesized that this result may be due to the generalization of behavior from reinforced cues to non-reinforced cues on this task. Therefore, we compared temporal learning rates from the PIP in the absence of reinforcement from our first experiment (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003e) to temporal learning on the PIP in presence of reinforcement from our second experiment (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003e). We found that the presence of reinforcement on an interval timing task increased temporal learning rates when compared to a task without reinforcement (Fig.\u0026nbsp;\u003cspan refid=\"Fig7\" class=\"InternalRef\"\u003e3\u003c/span\u003e). Interestingly, this increase in temporal learning rate was due to an increase in initial temporal error experienced on the reinforced version of the PIP when compared to the non-reinforced version of the PIP. To ensure these timing effects weren\u0026rsquo;t due to overall group level differences or additional cognitive load due to the introduction of reinforcement, we performed an supplementary experiment in 40 na\u0026iuml;ve participants that demonstrated that these interval timing effects were consistent in the face of reinforcers, and they scaled to the magnitude of reinforcement expected (\u003cb\u003eFig S5A\u003c/b\u003e; supplemental results).\u003c/p\u003e\u003cp\u003eAs the structure of the PIP used to measure interval timing in this study is similar to instrumental conditioning paradigms, it is possible that temporal learning is being accelerated through RPE-mediated mechanisms. Prior to this study, the role of reinforcers as a teaching signal, specifically for interval timing, was unclear. Therefore, we investigated whether RPEs associated with interval timing could help explain differences in temporal learning and temporal errors generated in the context of different valence of reinforcement. Our results show that RPEs on previous trials are associated with temporal performance prediction errors (PPEs) on the current trial for positively reinforced 1000ms cues (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e), which suggests that RPEs are reinforcing the poor performance exhibited on the previous trial. Previous research investigating the role of dopamine in interval timing have led some to propose that reinforcement learning models could be used to explain the dopamine response to interval timing (Petter, et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Mikhael \u0026amp; Gershman \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2019\u003c/span\u003e). As delaying or emitting a reinforcer has been shown to cause the silencing of the dopamine response at the expected time of reward, it has been hypothesized that the dopamine response may track temporal errors as part of or in addition to errors in reward valuation (Hollerman \u0026amp; Schultz \u003cspan citationid=\"CR13\" class=\"CitationRef\"\u003e1998\u003c/span\u003e). RPE-driven phasic dopamine signals at the outset of each trial and at the end of each trial may be one possible explanation of this increased temporal error but also increased learning rate in the context of a task with reinforcement.\u003c/p\u003e\u003cp\u003eOur study provides evidence that the presence of reinforcement increases the rate of temporal learning for intervals of time of 1000ms in duration (\u003cb\u003eFig S3A\u003c/b\u003e), but not 3000ms or 5000ms durations (\u003cb\u003eFig S3B-C\u003c/b\u003e). This result is aligned with other research, as different interval timing behavioral effects have been described when comparing durations of less than and greater than approximately 1200ms, with individual differences in the breakpoint (Artieda, et al., \u003cspan citationid=\"CR1\" class=\"CitationRef\"\u003e1992\u003c/span\u003e; Koch, et al., 2008). It has been hypothesized that different dopaminergic circuitry may be involved in the perception of durations of time less than and greater than approximately 1200ms (Hinton \u0026amp; Meck, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2004\u003c/span\u003e). While dopaminergic action in the striatum is thought to play a role in the perception of time of intervals greater than approximately 350ms, the prefrontal cortex and other \u0026ldquo;decision-making centers\u0026rdquo; are believed to be engaged when perceiving intervals greater than about 1200ms, as perceiving longer intervals of time is thought to require working memory and temporal planning processes (Smith \u0026amp; Jonides, \u003cspan citationid=\"CR36\" class=\"CitationRef\"\u003e1999\u003c/span\u003e; Hinton \u0026amp; Meck, \u003cspan citationid=\"CR12\" class=\"CitationRef\"\u003e2004\u003c/span\u003e; Meck \u003cspan citationid=\"CR20\" class=\"CitationRef\"\u003e2005\u003c/span\u003e). Our results provide additional evidence of a behavioral difference between timing intervals 1000ms in duration and intervals of greater than 3000ms in duration, as well as may provide some insight into the potential underlying dopaminergic mechanism for intervals of a 1000ms in duration.\u003c/p\u003e\u003cp\u003eOur results demonstrate a significant effect of learning signals that are shared between instrumental learning paradigms and methods used to investigate interval timing in animal models. In humans, the peak-interval procedure that we used to study interval timing can be instructed and participant performance can be measured, allowing for initial temporal learning effects to be observed. In the TDRL framework, there are two key variables that contribute to the calculation of a reward prediction error. We investigate the impact these signals have on temporal learning by comparing temporal learning behavior when these signals are present (PIP-r) versus absent (PIP-n). In TDRL, cues that predict a future possible reward can generate a reward prediction error in the absence of a concurrent reward. It was recently shown that TD-RPEs can elicit a phasic increase in dopamine levels in human striatum (Sands et al., \u003cspan citationid=\"CR34\" class=\"CitationRef\"\u003e2023\u003c/span\u003e). Together this suggests that the RPE effects observed here may be due to hypothesized phasic increases in dopamine. Interestingly, Sands et al., also showed that dopamine levels rose to the absence of punishment when a punishment was expected (a punishment prediction error) but fell when the punishment was worse than expected. In the current results, the RPE on expected punishment trials failed to reinforce poor performance but also did not significantly enhance temporal learning. In the context of rewards and punishments, cues that predict neither (the neutral cues in PIP-r) led to significantly faster temporal learning than neutral cues in PIP-n. One possible explanation would be that the neutral cues in the context of PIP-r gained an expected value that was non-punishing and therefore surprisingly good (better than expected relative to a possibly punishing trial) even though they were not reinforced positively. We did not model the TDRL-RPE in the current work due to significant challenges to modeling the peak-interval procedures without making significant and currently unresolvable assumptions about how the state-space of the task may be represented in the participants\u0026rsquo; brains.\u003c/p\u003e\u003cp\u003eTogether, our results suggest a hypothetical role of reinforcement learning mediated dopamine signals in time perception. Others have suggested such a role (Gershman, et al., \u003cspan citationid=\"CR10\" class=\"CitationRef\"\u003e2014\u003c/span\u003e; Petter, et al., \u003cspan citationid=\"CR25\" class=\"CitationRef\"\u003e2018\u003c/span\u003e; Mikhael \u0026amp; Gershman, \u003cspan citationid=\"CR21\" class=\"CitationRef\"\u003e2019\u003c/span\u003e; and Jakob, et al., \u003cspan citationid=\"CR14\" class=\"CitationRef\"\u003e2022\u003c/span\u003e) but have proposed models that deviate from the simplest and most well-established model for dopaminergic neuron activity and dopamine release (i.e., TDRL representations). Future work should be aimed at understanding whether the expected value and reward prediction error effects observed here in humans is indeed mediated by associated changes in dopamine in human striatum in line with the TDRL framework. Nevertheless, we have demonstrated that error signals, previously demonstrated to modulate phasic dopamine levels, can modulate timing behavior and temporal learning in paradoxical ways. Without directly measuring dopamine during these behaviors in the absence of confounding behavioral training, it will be difficult to tease out the role dopamine plays specifically in timing behaviors and temporal learning versus effects that are likely associated with instrumental learning paradigms requires to have non-human animals perform these behaviors. While dopamine, reward prediction errors, and timing behaviors are clearly interconnected, significant work remains to develop a formal model of the time we perceive.\u003c/p\u003e"},{"header":"Materials and Methods","content":"\u003cp\u003e\u003cb\u003eExperimental Design.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe recruited and consented two groups of 24 participants each (48 total) from the Winston-Salem, North Carolina region using methods approved by the Wake Forest University School of Medicine IRB (IRB00042265) for a one-hour study visit (\u003cb\u003eTable\u0026nbsp;1\u003c/b\u003e). All research was performed in accordance with relevant guidelines/regulations per Wake Forest University School of Medicine IRB, and informed consent was obtained from all participants in accordance with the Declaration of Helsinki. One participant was excluded from the study due to accidental data loss.\u003c/p\u003e\u003cp\u003eParticipants sat approximately 2 feet away from a Dell computer monitor and placed their dominant pointer finger on a button from a hand-held button box. Participants were then instructed on how to play the Peak Interval Procedure (PIP). The task was designed in Python 3 using the PyGame library. On initialization of the task, nine visual cues were randomly selected from a pool of 60 fractal images sourced from free online stock images. Each cue was assigned an interval of time (3 cues to each category: 1000ms, 3000ms, or 5000ms) (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA, Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). For the task that included reinforcements, cues were further divided into categories of reinforcement (win, loss, or nothing) (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). The maximum reinforcement amount was scaled to \u003cspan\u003e$\u003c/span\u003e3 on all trials on the PIP-r. The reinforcement was calculated using a linear decay of maximum reinforcement in relation to the error in the reproduction time (for example, a reproduction of 950ms for a 1000ms win trial would equate to a reinforcement of +\u003cspan\u003e$\u003c/span\u003e0.95 whereas a reproduction of 950ms for a 1000ms loss trial will equate to a reinforcement of -\u003cspan\u003e$\u003c/span\u003e0.05). Cues were randomly presented by placing each cue into an array 16 times and shuffling to determine the sequence of trials for a total of 144 trials.\u003c/p\u003e\u003cp\u003eStages of the task included: presentation, prompt, reproduction, and reinforcement. During the presentation stage, participants observed the presented cue for the assigned duration (1000ms, 3000ms, or 5000ms) (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA, \u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). Following a black 500ms screen, a \u0026lsquo;ready\u0026rsquo; prompt was displayed for 1000ms. After another 500ms black screen, the cue reappeared. Participants were instructed to use the hand-held button box to reproduce the duration of time by pressing the button when the cued duration elapsed. The cue disappeared when the button was pressed or after twice the cued duration passed. Immediately after, a monetary reinforcement or a hairpin cross was displayed based on the reinforcement type. After reinforcement, or if the cue type was neutral, the trial was complete. The cue disappeared, and a black screen was shown for an interval of time randomly drawn from a Poisson distribution with lambda equal to 1500ms. On average, experiments lasted\u0026thinsp;~\u0026thinsp;23 minutes per participant (\u003cb\u003eTable\u0026nbsp;1\u003c/b\u003e). At the end of the study visit, participants were compensated based on the study duration (\u003cspan\u003e$\u003c/span\u003e5 per 15 minutes), with a bonus based on task performance (ranging from \u003cspan\u003e$\u003c/span\u003e20 - \u003cspan\u003e$\u003c/span\u003e60 additional). Analysis was completed using R Studio.\u003c/p\u003e\u003cp\u003e\u003cb\u003eExperiment 1.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eTo characterize interval timing in response to predictable cues in the absence of reinforcement, we created the PIP-neutral (PIP-n), which presented participants with nine different cues, three of each were predictive of 1000ms, 3000ms, and 5000ms criterion durations, respectively (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eA). We recruited 24 participants to complete this task and measured temporal learning using learning curves (Fig.\u0026nbsp;\u003cspan refid=\"Fig5\" class=\"InternalRef\"\u003e1\u003c/span\u003eB\u003cb\u003e)\u003c/b\u003e. Learning curves were calculated for each participant based on absolute value of the temporal error for each cue type at each appearance, using Eq.\u0026nbsp;(1):\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eEq 1\u003c/strong\u003e\u003cp\u003eTemporal Accuracy = |RD-CD|\u003c/p\u003e\u003c/p\u003e\u003cp\u003eWhere RD is the participant\u0026rsquo;s reproduced duration and CD is the criterion duration. Mean absolute value of the error at each appearance was then fit with an exponential function to determine the rate of temporal learning overtime, using Eq.\u0026nbsp;(2):\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eEq 2\u003c/strong\u003e\u003cp\u003eY\u003csub\u003ei\u003c/sub\u003e = B\u003csub\u003e0\u003c/sub\u003e + B\u003csub\u003e1\u003c/sub\u003e*log(X\u003csub\u003ei\u003c/sub\u003e)\u003c/p\u003e\u003c/p\u003e\u003cp\u003eWhere Y is absolute error for each cue (i). X is each appearance of the cue (i). B\u003csub\u003e0\u003c/sub\u003e represents the intercept and measures the value where the line crosses the y-axis. B\u003csub\u003e1\u003c/sub\u003e is the slope of the curve and measures the rate of change in temporal error or the temporal learning rate.\u003c/p\u003e\u003cp\u003e\u003cb\u003eExperiment 2.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe modified the PIP to deliver reinforcements on 2/3rds of all trials immediately following the participant\u0026rsquo;s button press, in a task called the PIP-reinforced (PIP-r; Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eA). During the PIP-r, a reinforcement was a reward (monetary gain) or punishment (monetary loss) immediately following the reproduced duration. The monetary reinforcement structure was the amount of money a participant gained or lost (scaled to \u0026plusmn;\u003cspan\u003e$\u003c/span\u003e3) on a trial and was calculated in a linear fashion from the temporal accuracy of the participant on that trial (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eB). Temporal accuracy was calculated using Eq.\u0026nbsp;(1). Where RD was the participant\u0026rsquo;s reproduced duration and CD was the criterion duration (1000ms, 3000ms, or 5000ms). To ensure there were no effects of priors on performance, we recruited 24 naive participants to complete the PIP-r and measured their interval timing using learning curves (Fig.\u0026nbsp;\u003cspan refid=\"Fig6\" class=\"InternalRef\"\u003e2\u003c/span\u003eC).\u003c/p\u003e\u003cp\u003e\u003cb\u003eReward Prediction Errors.\u003c/b\u003e\u003c/p\u003e\u003cp\u003eWe calculated reward prediction errors as the difference between what participants might have expected to earn on a given cue and what they actually received (Fig.\u0026nbsp;\u003cspan refid=\"Fig8\" class=\"InternalRef\"\u003e4\u003c/span\u003e). Reward prediction error (RPE) was calculated using Eq.\u0026nbsp;(3):\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eEq 3\u003c/strong\u003e\u003cp\u003eRPE\u003csub\u003et\u003c/sub\u003e = V\u003csub\u003et\u003c/sub\u003e \u0026ndash;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\frac{1}{t-1}\\sum\\:_{1}^{t-1}{V}_{(t-1)}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\u003c/p\u003e\u003cp\u003eWhere t is the current trial of the same cue type (criterion duration x reinforcement type) and V is the monetary value of that trial. Performance prediction errors (PPE) were calculated based on the difference between the expected reproduced duration on a given cue and how participants actually responded to the cue using Eq.\u0026nbsp;(4):\u003c/p\u003e\u003cp\u003e\u003cstrong\u003eEq 4\u003c/strong\u003e\u003cp\u003ePPE\u003csub\u003et\u003c/sub\u003e = RD\u003csub\u003et\u003c/sub\u003e \u0026ndash;\u003cspan class=\"InlineEquation\"\u003e\u003cspan class=\"mathinline\"\u003e\\(\\:\\frac{1}{t-1}\\sum\\:_{1}^{t-1}{RD}_{(t-1)}\\)\u003c/span\u003e\u003c/span\u003e\u003c/p\u003e\u003c/p\u003e\u003cp\u003eWhere t is the current trial with the same cue type (criterion duration x reinforcement type) and RD is the reproduced duration on that trial.\u003c/p\u003e\u003cdiv id=\"Sec5\" class=\"Section2\"\u003e\u003ch2\u003eStatistical Analysis.\u003c/h2\u003e\u003cp\u003eSample size was determined by an a priori power analysis. A group-level statistical assessment of the relationship between mean differences in temporal accuracy from a previous study (DiMarco et al. \u003cspan citationid=\"CR7\" class=\"CitationRef\"\u003e2023\u003c/span\u003e) while anticipating a large (0.8) effect size at power level of 0.8 and p-value of 0.01 for a two-tailed t-test, required recruiting 14\u0026ndash;32 participants (per group). These estimates do not use sex or age as factors. Large outliers were removed from analysis.\u003c/p\u003e\u003cp\u003eStatistics were performed using R Studio (Posit Team \u003cspan citationid=\"CR26\" class=\"CitationRef\"\u003e2024\u003c/span\u003e). Mixed effects ANOVAs were used to compare cue types within and across tasks, accounting for repeated measures across the same subjects. Multiple pairwise t-tests with Bonferroni correction were used to subsequently determine significance between individual cue types while adjusting the significance level to account for increased type I error due to multiple comparisons. Generalized eta squared (ges) was used to determine effect size within each ANOVA comparison. Linear regression models were used to determine associations between RPEs and PPEs across reinforcement context.\u003c/p\u003e\u003c/div\u003e"},{"header":"Declarations","content":"\u003cp\u003e\u003cstrong\u003eAcknowledgments\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eWe would like to extend our gratitude to our human participants for volunteering for this study and for their commitment to research.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eFunding:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eNational Institutes of Health grant KL2TR00142\u003c/p\u003e\n\u003cp\u003eNational Institutes of Health grant R01 DA048096\u003c/p\u003e\n\u003cp\u003eNational Institutes of Health grant R01 MH121099\u003c/p\u003e\n\u003cp\u003eNational Institutes of Health grant R01 NS092701\u003c/p\u003e\n\u003cp\u003eNational Institutes of Health grant R01 MH1241\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eAuthor contributions:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eConceptualization: EKD, KTK\u003c/p\u003e\n\u003cp\u003eMethodology: EKD, ARS, KTK\u003c/p\u003e\n\u003cp\u003eInvestigation: EKD\u003c/p\u003e\n\u003cp\u003eVisualization: EKD\u003c/p\u003e\n\u003cp\u003eSupervision: KTK\u003c/p\u003e\n\u003cp\u003eWriting—original draft: EKD, KTK\u003c/p\u003e\n\u003cp\u003eWriting—review \u0026amp; editing: EKD, ARS, KTK\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eCompeting interests:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAuthors declare that they have no competing interests.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eData and materials availability:\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eAll data are available in the main text or the supplementary materials.\u003c/p\u003e"},{"header":"References","content":"\u003col\u003e\n\u003cli\u003eArtieda, J., Pastor, M. A., Lacruz, F., \u0026amp; Obeso, J. A. (1992). Temporal discrimination is abnormal in Parkinson\u0026apos;s disease. Brain: a journal of neurology, 115 Pt 1, 199\u0026ndash;210. https://doi.org/10.1093/brain/115.1.199\u003c/li\u003e\n\u003cli\u003eBalcı, F., \u0026amp; Freestone, D. (2020). The Peak Interval Procedure in Rodents: A Tool for Studying the Neurobiological Basis of Interval Timing and Its Alterations in Models of Human Disease. Bio-protocol, 10(17), e3735. https://doi.org/10.21769/BioProtoc.3735\u003c/li\u003e\n\u003cli\u003eBuhusi, C. V., \u0026amp; Meck, W. H. (2002). Differential effects of methamphetamine and haloperidol on the control of an internal clock. \u003cem\u003eBehavioral Neuroscience, 116\u003c/em\u003e(2), 291\u0026ndash;297. https://doi.org/10.1037/0735-7044.116.2.291\u003c/li\u003e\n\u003cli\u003eBuhusi, C. V., \u0026amp; Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. \u003cem\u003eNature reviews. Neuroscience\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e(10), 755\u0026ndash;765. https://doi.org/10.1038/nrn1764\u003c/li\u003e\n\u003cli\u003eChurch, R.M. (2012). Temporal Learning in Humans and Other Animals. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_66\u003c/li\u003e\n\u003cli\u003eCoull, J. T., Hwang, H. J., Leyton, M., \u0026amp; Dagher, A. (2012). Dopamine precursor depletion impairs timing in healthy volunteers by attenuating activity in putamen and supplementary motor area. The Journal of neuroscience: the official journal of the Society for Neuroscience, 32(47), 16704\u0026ndash;16715. https://doi.org/10.1523/JNEUROSCI.1258-12.2012\u003c/li\u003e\n\u003cli\u003eDiMarco, E. K., Sadibolova, R., Jiang, A., Liebenow, B., Jones, R. E., Haq, I. U., Siddiqui, M. S., Terhune, D. B., \u0026amp; Kishida, K. T. (2023). Time perception reflects individual differences in motor and non-motor symptoms of Parkinson\u0026apos;s disease. \u003cem\u003eParkinsonism \u0026amp; related disorders\u003c/em\u003e, \u003cem\u003e114\u003c/em\u003e, 105800. https://doi.org/10.1016/j.parkreldis.2023.105800\u003c/li\u003e\n\u003cli\u003eDrew, M. R., Fairhurst, S., Malapani, C., Horvitz, J. C., \u0026amp; Balsam, P. D. (2003). Effects of dopamine antagonists on the timing of two intervals. Pharmacology, biochemistry, and behavior, 75(1), 9\u0026ndash;15. https://doi.org/10.1016/s0091-3057(03)00036-4\u003c/li\u003e\n\u003cli\u003eFrank, M. J., \u0026amp; Fossella, J. A. (2011). Neurogenetics and pharmacology of learning, motivation, and cognition. Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology, 36(1), 133\u0026ndash;152. https://doi.org/10.1038/npp.2010.96\u003c/li\u003e\n\u003cli\u003eGershman, S. J., Moustafa, A. A., \u0026amp; Ludvig, E. A. (2014). Time representation in reinforcement learning models of the basal ganglia. Frontiers in computational neuroscience, 7, 194. https://doi.org/10.3389/fncom.2013.00194\u003c/li\u003e\n\u003cli\u003eGlimcher P. W. (2011). Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. \u003cem\u003eProceedings of the National Academy of Sciences of the United States of America\u003c/em\u003e, \u003cem\u003e108 Suppl 3\u003c/em\u003e(Suppl 3), 15647\u0026ndash;15654. https://doi.org/10.1073/pnas.1014269108\u003c/li\u003e\n\u003cli\u003eHinton, S. C., \u0026amp; Meck, W. H. (2004). Frontal-striatal circuitry activated by human peak-interval timing in the supra-seconds range. \u003cem\u003eBrain research. Cognitive brain research\u003c/em\u003e, \u003cem\u003e21\u003c/em\u003e(2), 171\u0026ndash;182. https://doi.org/10.1016/j.cogbrainres.2004.08.005\u003c/li\u003e\n\u003cli\u003eHollerman, J. R., \u0026amp; Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature neuroscience, 1(4), 304\u0026ndash;309. https://doi.org/10.1038/1124\u003c/li\u003e\n\u003cli\u003eJakob, A. M. V., Mikhael, J. G., Hamilos, A. E., Assad, J. A., \u0026amp; Gershman, S. J. (2022). Dopamine mediates the bidirectional update of interval timing. \u003cem\u003eBehavioral neuroscience\u003c/em\u003e, \u003cem\u003e136\u003c/em\u003e(5), 445\u0026ndash;452. https://doi.org/10.1037/bne0000529\u003c/li\u003e\n\u003cli\u003eKim, H. R., Malik, A. N., Mikhael, J. G., Bech, P., Tsutsui-Kimura, I., Sun, F., Zhang, Y., Li, Y., Watabe-Uchida, M., Gershman, S. J., \u0026amp; Uchida, N. (2020). A Unified Framework for Dopamine Signals across Timescales. \u003cem\u003eCell\u003c/em\u003e, \u003cem\u003e183\u003c/em\u003e(6), 1600\u0026ndash;1616.e25. https://doi.org/10.1016/j.cell.2020.11.013\u003c/li\u003e\n\u003cli\u003eKishida K. T., \u0026amp; Sands L. P. (2021). A Dynamic Affective Core to Bind the Contents, Context, and Value of Conscious Experience. Affect Dynamics, 293-328:8.\u003c/li\u003e\n\u003cli\u003eKoch, G., Costa, A., Brusa, L., Peppe, A., Gatto, I., Torriero, S., Gerfo, E. L., Salerno, S., Oliveri, M., Carlesimo, G. A., \u0026amp; Caltagirone, C. (2008). Impaired reproduction of second but not millisecond time intervals in Parkinson\u0026apos;s disease. Neuropsychologia, 46(5), 1305\u0026ndash;1313. https://doi.org/10.1016/j.neuropsychologia.2007.12.005\u003c/li\u003e\n\u003cli\u003eMacInnis, M.L.M., \u0026amp; Guilhardi, P. (2006). Basic interval discrimination procedures. In M. A. Anderson (Ed.), Tasks and Techniques: A Sampling of Methodologies for the Investigation of Animal Learning, Behavior, and Cognition, pp. 233-244. Hauppauge, NY: Nova Science Publishers. SBN 1-60021-126-7.\u003c/li\u003e\n\u003cli\u003eMeck W. H. (1996). Neuropharmacology of timing and time perception. Brain research. Cognitive brain research, 3(3-4), 227\u0026ndash;242. https://doi.org/10.1016/0926-6410(96)00009-2\u003c/li\u003e\n\u003cli\u003eMeck W. H. (2005). Neuropsychology of timing and time perception. \u003cem\u003eBrain and cognition\u003c/em\u003e, \u003cem\u003e58\u003c/em\u003e(1), 1\u0026ndash;8. https://doi.org/10.1016/j.bandc.2004.09.004\u003c/li\u003e\n\u003cli\u003eMikhael, J. G., \u0026amp; Gershman, S. J. (2019). Adapting the flow of time with dopamine. \u003cem\u003eJournal of neurophysiology\u003c/em\u003e, \u003cem\u003e121\u003c/em\u003e(5), 1748\u0026ndash;1760. https://doi.org/10.1152/jn.00817.2018\u003c/li\u003e\n\u003cli\u003eMontague, P. R., Hyman, S. E., \u0026amp; Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. \u003cem\u003eNature\u003c/em\u003e, \u003cem\u003e431\u003c/em\u003e(7010), 760\u0026ndash;767. https://doi.org/10.1038/nature03015\u003c/li\u003e\n\u003cli\u003eMontague, P. R., Dayan, P., \u0026amp; Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. \u003cem\u003eThe Journal of neuroscience: the official journal of the Society for Neuroscience\u003c/em\u003e, \u003cem\u003e16\u003c/em\u003e(5), 1936\u0026ndash;1947. https://doi.org/10.1523/JNEUROSCI.16-05-01936.1996\u003c/li\u003e\n\u003cli\u003ePaton, J. J., \u0026amp; Buonomano, D. V. (2018). The Neural Basis of Timing: Distributed Mechanisms for Diverse Functions. Neuron, 98(4), 687\u0026ndash;705. https://doi.org/10.1016/j.neuron.2018.03.045\u003c/li\u003e\n\u003cli\u003ePetter, E. A., Gershman, S. J., \u0026amp; Meck, W. H. (2018). Integrating Models of Interval Timing and Reinforcement Learning. \u003cem\u003eTrends in cognitive sciences\u003c/em\u003e, \u003cem\u003e22\u003c/em\u003e(10), 911\u0026ndash;922. https://doi.org/10.1016/j.tics.2018.08.004\u003c/li\u003e\n\u003cli\u003ePosit team (2024). RStudio: Integrated Development Environment for R. Posit Software, PBC, Boston, MA. URL http://www.posit.co/.\u003c/li\u003e\n\u003cli\u003eRakitin, B. C., Gibbon, J., Penney, T. B., Malapani, C., Hinton, S. C., \u0026amp; Meck, W. H. (1998). Scalar expectancy theory and peak-interval timing in humans. Journal of experimental psychology. Animal behavior processes, 24(1), 15\u0026ndash;33. https://doi.org/10.1037//0097-7403.24.1.15\u003c/li\u003e\n\u003cli\u003eRammsayer T. H. (1999). Neuropharmacological evidence for different timing mechanisms in humans. The Quarterly journal of experimental psychology. B, Comparative and physiological psychology, 52(3), 273\u0026ndash;286. https://doi.org/10.1080/713932708\u003c/li\u003e\n\u003cli\u003eRammsayer, T. H., \u0026amp; Troche, S. J. (2014). In search of the internal structure of the processes underlying interval timing in the sub-second and the second range: A confirmatory factor analysis approach. \u003cem\u003eActa Psychologica, 147,\u003c/em\u003e 68\u0026ndash;74. https://doi.org/10.1016/j.actpsy.2013.05.004\u003c/li\u003e\n\u003cli\u003eRammsayer, T. H., Borter, N., \u0026amp; Troche, S. J. (2015). Visual-auditory differences in duration discrimination of intervals in the subsecond and second range. \u003cem\u003eFrontiers in psychology\u003c/em\u003e, \u003cem\u003e6\u003c/em\u003e, 1626. https://doi.org/10.3389/fpsyg.2015.01626\u003c/li\u003e\n\u003cli\u003eRescorla R. A., Wagner A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. \u003cem\u003eClassical conditioning, Current research and theory\u003c/em\u003e, \u003cem\u003e2\u003c/em\u003e, 64-69.\u003c/li\u003e\n\u003cli\u003eSadibolova, R., Monaldi, L., \u0026amp; Terhune, D. B. (2022). A proxy measure of striatal dopamine predicts individual differences in temporal precision. Psychonomic bulletin \u0026amp; review, 29(4), 1307\u0026ndash;1316. https://doi.org/10.3758/s13423-022-02077-1\u003c/li\u003e\n\u003cli\u003eSadibolova, R., DiMarco, E. K., Jiang, A., Maas, B., Tatter, S. B., Laxton, A., Kishida, K. T., \u0026amp; Terhune, D. B. (2024). Sub-second and multi-second dopamine dynamics underlie variability in human time perception. \u003cem\u003emedRxix: the preprint server for health sciences\u003c/em\u003e, 2024.02.09.24302276. https://doi.org/10.1101/2024.02.09.24302276\u003c/li\u003e\n\u003cli\u003eSands, L. P., Jiang, A., Liebenow, B., DiMarco, E., Laxton, A. W., Tatter, S. B., Montague, P. R., \u0026amp; Kishida, K. T. (2023). Subsecond fluctuations in extracellular dopamine encode reward and punishment prediction errors in humans. \u003cem\u003eScience advances\u003c/em\u003e, \u003cem\u003e9\u003c/em\u003e(48), eadi4927. https://doi.org/10.1126/sciadv.adi4927\u003c/li\u003e\n\u003cli\u003eSchultz, W., Dayan, P., \u0026amp; Montague, P. R. (1997). A neural substrate of prediction and reward. \u003cem\u003eScience (New York, N.Y.)\u003c/em\u003e, \u003cem\u003e275\u003c/em\u003e(5306), 1593\u0026ndash;1599. https://doi.org/10.1126/science.275.5306.1593\u003c/li\u003e\n\u003cli\u003eSmith, E. E., \u0026amp; Jonides, J. (1999). Storage and executive processes in the frontal lobes. \u003cem\u003eScience (New York, N.Y.)\u003c/em\u003e, \u003cem\u003e283\u003c/em\u003e(5408), 1657\u0026ndash;1661. https://doi.org/10.1126/science.283.5408.1657\u003c/li\u003e\n\u003cli\u003eSutton R, Barto A. Reinforcement Learning: An Introduction. The MIT Press, First edition, (1998).\u003c/li\u003e\n\u003cli\u003eTerhune, D. B., Sullivan, J. G., \u0026amp; Simola, J. M. (2016). Time dilates after spontaneous blinking. \u003cem\u003eCurrent biology: CB\u003c/em\u003e, \u003cem\u003e26\u003c/em\u003e(11), R459\u0026ndash;R460. https://doi.org/10.1016/j.cub.2016.04.010\u003c/li\u003e\n\u003c/ol\u003e"}],"fulltextSource":"","fullText":"","funders":[],"hasAdminPriorityOnWorkflow":false,"hasManuscriptDocX":true,"hasOptedInToPreprint":true,"hasPassedJournalQc":"","hasAnyPriority":false,"hideJournal":false,"highlight":"","institution":"","isAcceptedByJournal":false,"isAuthorSuppliedPdf":false,"isDeskRejected":"","isHiddenFromSearch":false,"isInQc":false,"isInWorkflow":false,"isPdf":false,"isPdfUpToDate":true,"isWithdrawnOrRetracted":false,"journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true},"keywords":"","lastPublishedDoi":"10.21203/rs.3.rs-6968461/v1","lastPublishedDoiUrl":"https://doi.org/10.21203/rs.3.rs-6968461/v1","license":{"name":"CC BY 4.0","url":"https://creativecommons.org/licenses/by/4.0/"},"manuscriptAbstract":"\u003cp\u003eTime perception is often investigated using instrumental paradigms where reinforcement learning and associated dopaminergic processes have modulatory effects. Interval timing, which includes the judgment of relatively short intervals of time (ranging from milliseconds to minutes), has been shown to be modulated by manipulations of striatal dopamine. In reinforcement learning theory, the \u0026lsquo;expected value of reward\u0026rsquo; (EV) and \u0026lsquo;reward prediction errors\u0026rsquo; (RPEs) are key variables that explain striatal dopaminergic signals of reward processing during instrumental learning. Despite potential common dopaminergic underpinnings, the underlying connection between reinforcement learning processes and interval timing remains relatively underexplored. Herein, we investigated the impact of EV and RPEs on the human reproduction of 1000ms, 3000ms, and 5000ms intervals of time. Our results demonstrate that RPEs \u0026ndash; specifically about rewards and not punishments \u0026ndash; appear to reinforce performance errors, which effectively interfere with the rate at which reinforced 1000ms intervals \u0026ndash; but not 3000ms and 5000ms intervals \u0026ndash; are learned. The results of these experiments help clarify the role reinforcements play in interval timing, as well as give insight into the hypothetical mechanisms underlying time perception and the potential shared relationship with reinforcement learning processes.\u003c/p\u003e","manuscriptTitle":"Expected reward value and reward prediction errors reinforce but also interfere with human time perception","msid":"","msnumber":"","nonDraftVersions":[{"code":1,"date":"2025-07-24 18:00:27","doi":"10.21203/rs.3.rs-6968461/v1","editorialEvents":[{"type":"communityComments","content":0},{"type":"decision","content":"Revision requested","date":"2026-02-11T09:16:37+00:00","index":"","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-10T02:39:57+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-10T01:14:34+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2026-02-09T10:32:50+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"173614102986778309425990735431752898651","date":"2026-02-01T22:49:35+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"287142211853632404322078740577449291651","date":"2026-02-01T09:54:18+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"76048034186933175905011392592274662403","date":"2026-01-31T19:42:15+00:00","index":"hide","fulltext":""},{"type":"editorInvitedReview","content":"","date":"2025-10-07T11:47:59+00:00","index":"hide","fulltext":""},{"type":"reviewerAgreed","content":"205971215962363644736065892990503542778","date":"2025-09-24T14:50:53+00:00","index":"hide","fulltext":""},{"type":"reviewersInvited","content":"","date":"2025-07-22T19:25:48+00:00","index":"","fulltext":""},{"type":"editorAssigned","content":"","date":"2025-07-22T16:52:32+00:00","index":"","fulltext":""},{"type":"editorInvited","content":"","date":"2025-06-26T12:07:15+00:00","index":"","fulltext":""},{"type":"checksComplete","content":"","date":"2025-06-26T03:33:01+00:00","index":"","fulltext":""},{"type":"submitted","content":"Scientific Reports","date":"2025-06-24T18:58:21+00:00","index":"","fulltext":""}],"status":"published","journal":{"display":true,"email":"[email protected]","identity":"scientific-reports","isNatureJournal":false,"hasQc":true,"allowDirectSubmit":false,"externalIdentity":"scirep","sideBox":"Learn more about [Scientific Reports](http://www.nature.com/srep/)","snPcode":"","submissionUrl":"","title":"Scientific Reports","twitterHandle":"","acdcEnabled":true,"dfaEnabled":true,"editorialSystem":"stoa","reportingPortfolio":"Scientific Reports","inReviewEnabled":true,"inReviewRevisionsEnabled":true}}],"origin":"","ownerIdentity":"0bed219c-5097-4c82-b180-ffa5c5d5de1e","owner":[],"postedDate":"July 24th, 2025","published":true,"recentEditorialEvents":[],"rejectedJournal":[],"revision":"","amendment":"","status":"in-revision","subjectAreas":[{"id":52020862,"name":"Biological sciences/Neuroscience/Cognitive neuroscience"},{"id":52020863,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Decision"},{"id":52020864,"name":"Biological sciences/Neuroscience/Cognitive neuroscience/Perception"}],"tags":[],"updatedAt":"2026-05-07T07:55:20+00:00","versionOfRecord":[],"versionCreatedAt":"2025-07-24 18:00:27","video":"","vorDoi":"","vorDoiUrl":"","workflowStages":[]},"version":"v1","identity":"rs-6968461","journalConfig":"researchsquare"},"__N_SSP":true},"page":"/article/[identity]/[[...version]]","query":{"redirect":"/article/rs-6968461","identity":"rs-6968461","version":["v1"]},"buildId":"8U1c8b4HqxoKbykW_rLl7","isFallback":false,"isExperimentalCompile":false,"dynamicIds":[84888],"gssp":true,"scriptLoader":[]}