Surprise! Dopamine signals mix action, value and error

Collins, Anne; Frank, Michael J.

doi:10.1038/nn.4207

Cited by 31 publications

(23 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Correspondingly, we performed formal model comparisons using a likelihood ratio test and found that a model including an interaction between RPE valence and surprise explained behavior better than a model including only the signed RPE (Stay/switch behavior: (2) = 36.24, P < 0.001; Delta RTs: (2) = 18.81, P < 0.001). This result is in line with the growing consensus that separate learning systems are at play in the human brain 28 .…”

Section: Resultssupporting

confidence: 89%

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Fouragnan

Queirazza

Retzler

et al. 2017

Sci Rep

View full text Add to dashboard Cite

Reward learning depends on accurate reward associations with potential choices. These associations can be attained with reinforcement learning mechanisms using a reward prediction error (RPE) signal (the difference between actual and expected rewards) for updating future reward expectations. Despite an extensive body of literature on the influence of RPE on learning, little has been done to investigate the potentially separate contributions of RPE valence (positive or negative) and surprise (absolute degree of deviation from expectations). Here, we coupled single-trial electroencephalography with simultaneously acquired fMRI, during a probabilistic reversal-learning task, to offer evidence of temporally overlapping but largely distinct spatial representations of RPE valence and surprise. Electrophysiological variability in RPE valence correlated with activity in regions of the human reward network promoting approach or avoidance learning. Electrophysiological variability in RPE surprise correlated primarily with activity in regions of the human attentional network controlling the speed of learning. Crucially, despite the largely separate spatial extend of these representations our EEG-informed fMRI approach uniquely revealed a linear superposition of the two RPE components in a smaller network encompassing visuo-mnemonic and reward areas. Activity in this network was further predictive of stimulus value updating indicating a comparable contribution of both signals to reward learning.

show abstract

Section: Resultssupporting

confidence: 89%

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Fouragnan

Queirazza

Retzler

et al. 2017

Sci Rep

View full text Add to dashboard Cite

show abstract

“…Lesion and imaging studies suggest, for example, the costs related to effort versus delay different valuation systems (Prevost et al 2010, Rudebeck et al 2006) and produce distinct (and often uncorrelated) discounting behavior (Klein-Flügge et al 2015). Moreover, recent work has found that contrary to predictions based on a utility model of dopaminergic activity, DA linked reward signaling in the striatum is heavily influenced by whether action is necessary to get a reward (Collins & Frank 2016, Syed et al 2016). Consequently, the identification of neural signals that appear to track a pure utility signal in one type of experimental design (e.g., when rewards of different magnitudes all require some action to acquire), may fail to generalize to other paradigms.…”

Section: A Taxonomy Of Reward Processesmentioning

confidence: 77%

Reward Processing, Neuroeconomics, and Psychopathology

Zald

Treadway

2017

Annu. Rev. Clin. Psychol.

144

112

View full text Add to dashboard Cite

Abnormal reward processing is a prominent transdiagnostic feature of psychopathology. The present review provides a framework for considering the different aspects of reward processing and their assessment and highlight recent insights from the field of neuroeconomics that may aid in understanding these processes. Although altered reward processing in psychopathology has often been treated as a general hypo- or hyper-responsivity to reward, increasing data indicate that a comprehensive understanding of reward dysfunction requires characterization within more specific reward processing domains, including subjective valuation, discounting, hedonics, reward anticipation and facilitation, and reinforcement learning. As such, more nuanced models of the nature of these abnormalities are needed. We describe several processing abnormalities capable of producing the types of selective alterations in reward related behavior observed in different forms of psychopathology, including (mal)adaptive scaling and anchoring, dysfunctional weighting of reward and cost variables, completion between valuation systems, and positive prediction error signaling.

show abstract

“…Even more recently, a study combining reinforcement learning with both go and no-go responses found that the strength of DAergic RPE signaling in the striatum was heavily moderated by whether go or nogo responses were required to harvest a reward following a reward-predicting cue (Syed et al, 2016). Expanding on these results, Hamid et al (2016) used extended analysis of striatal DA release over repeated trials to show that DA ramps encoded an expected value of action, whereas short-term phasic bursts and dips reflected prediction error updates to these expected value estimates (Collins and Frank, 2016).…”

Section: Normal and Abnormal Da Function In The Context Of Motivationmentioning

confidence: 99%

Inflammation Effects on Motivation and Motor Activity: Role of Dopamine

Felger

Treadway

2016

Neuropsychopharmacol

317

237

View full text Add to dashboard Cite

Motivational and motor deficits are common in patients with depression and other psychiatric disorders, and are related to symptoms of anhedonia and motor retardation. These deficits in motivation and motor function are associated with alterations in corticostriatal neurocircuitry, which may reflect abnormalities in mesolimbic and mesostriatal dopamine (DA). One pathophysiologic pathway that may drive changes in DAergic corticostriatal circuitry is inflammation. Biomarkers of inflammation such as inflammatory cytokines and acute-phase proteins are reliably elevated in a significant proportion of psychiatric patients. A variety of inflammatory stimuli have been found to preferentially target basal ganglia function to lead to impaired motivation and motor activity. Findings have included inflammation-associated reductions in ventral striatal neural responses to reward anticipation, decreased DA and DA metabolites in cerebrospinal fluid, and decreased availability, and release of striatal DA, all of which correlated with symptoms of reduced motivation and/or motor retardation. Importantly, inflammation-associated symptoms are often difficult to treat, and evidence suggests that inflammation may decrease DA synthesis and availability, thus circumventing the efficacy of standard pharmacotherapies. This review will highlight the impact of administration of inflammatory stimuli on the brain in relation to motivation and motor function. Recent data demonstrating similar relationships between increased inflammation and altered DAergic corticostriatal circuitry and behavior in patients with major depressive disorder will also be presented. Finally, we will discuss the mechanisms by which inflammation affects DA neurotransmission and relevance to novel therapeutic strategies to treat reduced motivation and motor symptoms in patients with high inflammation.

show abstract

Surprise! Dopamine signals mix action, value and error

Abstract: Two studies invite us to reconsider the nature of striatal dopamine signals. Accumbens dopamine appears to signal the value of overt action and prediction errors arise from deviations in these signals.

Cited by 31 publications

References 15 publications

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Spatiotemporal neural characterization of prediction error valence and surprise during reward learning in humans

Reward Processing, Neuroeconomics, and Psychopathology

Inflammation Effects on Motivation and Motor Activity: Role of Dopamine

Contact Info

Product

Resources

About