Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum

Takahashi, Yuji K.; Langdon, Angela J.; Niv, Yaron; Schoenbaum, Geoffrey

doi:10.1016/j.neuron.2016.05.015

Cited by 104 publications

(129 citation statements)

References 67 publications

Supporting

Mentioning

120

Contrasting

Order By: Relevance

“…In addition, we found striosome to be a key brain nucleus which remembers the timing of previous rewards and encodes the predicted rewards. In fact, there are recent experimental works [51] that support our model prediction.…”

Section: Discussionsupporting

confidence: 77%

Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward‐Based Phasic Neural Responses

2018

View full text Add to dashboard Cite

Phasic activity of dopaminergic (DA) neurons in the ventral tegmental area or substantia nigra compacta (VTA/SNc) has been suggested to encode reward-prediction error signal for reinforcement learning. Recent studies have shown that the lateral habenula (LHb) neurons exhibit a similar response, but for nonrewarding or punishment signals. Hence, the transient signaling role of LHb neurons is opposite that of DA neurons and also that of several other brain nuclei such as the border region of the globus pallidus internal segment (GPb) and the rostral medial tegmentum (RMTg). Previous theoretical models have investigated the neural circuit mechanism underlying reward-based phasic activity of DA neurons, but the feasibility of a larger neural circuit model to account for the observed reward-based phasic activity in other brain nuclei such as the LHb has yet to be shown. Here, we propose a largescale neural circuit model and show that parallel excitatory and inhibitory pathways underlie the learned neural responses across multiple brain regions. Specifically, the model can account for the phasic neural activity observed in the GPb, LHb, RMTg, and VTA/SNc. Based on sensitivity analysis, the model is found to be robust against changes in the overall neural connectivity strength. The model also predicts that striosomes play a key role in the phasic activity of VTA/SNc and LHb neurons by encoding previous and expected rewards. Taken together, our model identifies the important role of parallel neural circuit pathways in accounting for phasic activity across multiple brain areas during reward and punishment processing.

show abstract

Section: Discussionsupporting

confidence: 77%

Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward‐Based Phasic Neural Responses

2018

View full text Add to dashboard Cite

show abstract

“…In rodents, NAc core lesions increase delay discounting (Cardinal et al, 2001) and impair performance when reward is contingent on withholding a response for a fixed period of time (Pothuizen et al, 2005). VS lesions also disrupt the temporal specificity of dopaminergic RPEs induced by changes in reward timing (Takahashi et al, 2016). Altogether, these findings suggest VS regulates temporal integration of reward information to guide choice behavior.…”

Section: Discussionmentioning

confidence: 99%

Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning

Costa

Monte

Lucas

et al. 2016

Neuron

135

127

View full text Add to dashboard Cite

Summary Reinforcement learning (RL) theories posit that dopaminergic signals are integrated within the striatum to associate choices with outcomes. Often overlooked is that the amygdala also receives dopaminergic input and is involved in Pavlovian processes that influence choice behavior. To determine the relative contributions of the ventral striatum (VS) and amygdala to appetitive RL we tested rhesus macaques with VS or amygdala lesions on deterministic and stochastic versions of a two-arm bandit reversal learning task. When learning was characterized with a RL model relative to controls, amygdala lesions caused general decreases in learning from positive feedback and choice consistency. By comparison, VS lesions only affected learning in the stochastic task. Moreover, the VS lesions hastened the monkeys’ choice reaction times, which emphasized a speed-accuracy tradeoff that accounted for errors in deterministic learning. These results update standard accounts of RL by emphasizing distinct contributions of the amygdala and VS to RL.

show abstract

“…Dopamine neurons were identified via a waveform analysis used and validated by us and others previously (Jin & Costa, 2010; Y. S. Jo, J. Lee, & S. J. Mizumori, 2013; Roesch, Calu, & Schoenbaum, 2007; Takahashi et al, 2016; Y. K. Takahashi et al, 2011).…”

Section: Methodsmentioning

confidence: 99%

“…Here, in a new analysis that takes advantage of previously described data (Takahashi, Langdon, Niv, & Schoenbaum, 2016; Y. K. Takahashi et al, 2011), we test whether dopaminergic error signals in rats are influenced by inferential predictions of the sort highlighted in the aforementioned study in monkeys (Bromberg-Martin et al, 2010), and whether such effects depend on the OFC.…”

Section: Introductionmentioning

confidence: 99%

Effects of inference on dopaminergic prediction errors depend on orbitofrontal processing.

Takahashi¹,

Stalnaker²,

Roesch³

et al. 2017

Behavioral Neuroscience

Self Cite

View full text Add to dashboard Cite

Dopaminergic reward prediction errors in monkeys reflect inferential reward predictions that well-trained animals can make when associative rules change. Here, in a new analysis of previously described data, we test whether dopaminergic error signals in rats are influenced by inferential predictions and whether such effects depend on the orbitofrontal cortex (OFC). Dopamine neurons were recorded from controls or rats with ipsilateral OFC lesions during performance of a choice task in which odor cues signaled the availability of sucrose reward in two wells. To induce prediction errors, we manipulated either the timing or number of rewards delivered in each well across blocks of trials. Importantly, a change in reward at one well predicted a change in reward at the other on later trials. We compared behavior and neural activity on trials when such inference was possible versus trials involving the same reward change when inference was not possible. Rats responded faster when they could infer an increase in reward compared to when the same reward was coming but they could not infer a change. This inferential prediction was reflected in the firing of dopamine neurons in controls, which changed less to unexpected delivery (or omission) of reward and more to the new high-value cue on inference versus non-inference trials. These effects were absent in dopamine neurons recorded in rats with ipsilateral OFC lesions. Thus, dopaminergic error signals recorded in rats are influenced by both experiential and inferential reward predictions, and the effects of inferential predictions depend on OFC.

show abstract

Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum

Cited by 104 publications

References 67 publications

Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward‐Based Phasic Neural Responses

Parallel Excitatory and Inhibitory Neural Circuit Pathways Underlie Reward‐Based Phasic Neural Responses

Amygdala and Ventral Striatum Make Distinct Contributions to Reinforcement Learning

Effects of inference on dopaminergic prediction errors depend on orbitofrontal processing.

Contact Info

Product

Resources

About