The Dopaminergic Midbrain Encodes the Expected Certainty about Desired Outcomes

Schwartenbeck, Philipp; FitzGerald, Thomas H. B.; Mathys, Christoph; Dolan, Raymond J.; Friston, Karl J.

doi:10.1093/cercor/bhu159

Cited by 193 publications

(218 citation statements)

References 75 publications

Supporting

Mentioning

204

Contrasting

Order By: Relevance

“…Prediction errors were compared using paired t tests. To assess the effect of effort and reward PEs on SN/VTA, we used the same generalized linear model applying small-volume FWE correction (uncorrected threshold P < 0.001) based on our anatomical SN/VTA mask (see below), similar to previous studies (e.g., 92,93).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals

Hauser

Eldar

Dolan

2017

Proc. Natl. Acad. Sci. U.S.A.

122

127

View full text Add to dashboard Cite

Optimal decision making mandates organisms learn the relevant features of choice options. Likewise, knowing how much effort we should expend can assume paramount importance. A mesolimbic network supports reward learning, but it is unclear whether other choice features, such as effort learning, rely on this same network. Using computational fMRI, we show parallel encoding of effort and reward prediction errors (PEs) within distinct brain regions, with effort PEs expressed in dorsomedial prefrontal cortex and reward PEs in ventral striatum. We show a common mesencephalic origin for these signals evident in overlapping, but spatially dissociable, dopaminergic midbrain regions expressing both types of PE. During action anticipation, reward and effort expectations were integrated in ventral striatum, consistent with a computation of an overall net benefit of a stimulus. Thus, we show that motivationally relevant stimulus features are learned in parallel dopaminergic pathways, with formation of an integrated utility signal at choice. effort prediction errors | reward prediction errors | apathy | substantia nigra/ventral tegmental area | dorsomedial prefrontal cortex O rganisms need to make energy-efficient decisions to maximize benefits and minimize costs, a tradeoff exemplified in effort expenditure (1-3). A key example occurs during foraging, where an overestimation of effort can lead to inaction and starvation (4), whereas underestimation of effort can result in persistent failure, as exemplified in the myth of Sisyphus (5).In a naturalistic environment, we often simultaneously learn about success in expending sufficient effort into an action as well as the reward we obtain from this same action. The reward outcomes that signal success and failure of an action are usually clear, although the effort necessary to attain success is often less transparent. Only by repeatedly experiencing success and failure is it possible to acquire an estimate of an optimal level of effort needed to succeed, without unnecessary waste of energy. This type of learning is important in contexts as diverse as foraging, hunting, and harvesting (6-8). Hull in his "law of less work" proposed that organisms "gradually learn" how to minimize effort expenditure (9). Surprisingly, we know little regarding the neurocognitive mechanisms that guide this form of simultaneous learning about reward and effort.A mesolimbic dopamine system encodes a teaching signal tethered to prediction of reward outcomes (10, 11). These reward prediction errors (PEs) arise from dopaminergic neurons in substantia nigra and ventral tegmental area (SN/VTA) and are broadcast to ventral striatum (VS) to mediate reward-related adaptation and learning (12, 13). Dopamine is also thought to provide a motivational signal (14-18), while dopaminergic deficits in rodents impair how effort and reward are arbitrated (1, 4, 19). The dorsomedial prefrontal cortex (dmPFC; spanning presupplementary motor area [pre-SMA] and dorsal anterior cingulate cortex [dACC]) is a candidate substrate...

show abstract

Section: Methodsmentioning

confidence: 99%

“…We used the mean structural MT image where SN/ VTA can be easily distinguished from surrounding areas as a bright white stripe (45), similar to previous studies (92,93).…”

Section: Methodsmentioning

confidence: 99%

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals

Hauser

Eldar

Dolan

2017

Proc. Natl. Acad. Sci. U.S.A.

122

127

View full text Add to dashboard Cite

show abstract

“…Model-based fMRI has subsequently been applied to numerous domains of cognition, accommodating a diversity of modelling approaches and computational themes, such as reinforcement learning (RL) models of behaviour and Bayesian models of cognition (e.g., (D'Ardenne et al, 2013;Daw et al, 2006;Iglesias et al, 2013;Klein-Flügge et al, 2011;Schwartenbeck et al, 2015;Seymour et al, 2004;Vossel et al, 2015). A recent application of model-based fMRI has been the investigation of interactions between learning and decision-making processes which do or do not derive from an explicit model of the environment or task structure.…”

Section: Accepted Manuscriptmentioning

confidence: 99%

Computational neuroimaging strategies for single patient predictions

et al. 2017

View full text Add to dashboard Cite

Neuroimaging increasingly exploits machine learning techniques in an attempt to achieve clinically relevant single-subject predictions. An alternative to machine learning, which tries to establish predictive links between features of the observed data and clinical variables, is the deployment of computational models for inferring on the (patho)physiological and cognitive mechanisms that generate behavioural and neuroimaging responses. This paper discusses the rationale behind a computational approach to neuroimaging-based single-subject inference, focusing on its potential for characterising disease mechanisms in individual subjects and mapping these characterisations to clinical predictions. Following an overview of two main approaches - Bayesian model selection and generative embedding - which can link computational models to individual predictions, we review how these methods accommodate heterogeneity in psychiatric and neurological spectrum disorders, help avoid erroneous interpretations of neuroimaging data, and establish a link between a mechanistic, model-based approach and the statistical perspectives afforded by machine learning.

show abstract

“…Several studies in animals and humans have provided evidence that uncertainty or precision are encoded by slow fluctuations in DA neuron activity within and across trials (de Lafuente and Romo, 2011;Fiorillo et al, 2003;Schwartenbeck et al, 2014). It has been suggested (Corlett et al, 2011;Friston, 2005b) that DA and other neuromodulators may serve to implement the precision-weighting of PEs that arises from Bayesian models under Gaussian assumptions Mathys et al, 2011;Rao and Ballard, 1999) ( Figure 5).…”

Section: Neuronmentioning

confidence: 99%

Translational Perspectives for Computational Neuroimaging

Stephan

Iglesias

Heinzle

et al. 2015

Neuron

164

148

View full text Add to dashboard Cite

Functional neuroimaging has made fundamental contributions to our understanding of brain function. It remains challenging, however, to translate these advances into diagnostic tools for psychiatry. Promising new avenues for translation are provided by computational modeling of neuroimaging data. This article reviews contemporary frameworks for computational neuroimaging, with a focus on forward models linking unobservable brain states to measurements. These approaches-biophysical network models, generative models, and model-based fMRI analyses of neuromodulation-strive to move beyond statistical characterizations and toward mechanistic explanations of neuroimaging data. Focusing on schizophrenia as a paradigmatic spectrum disease, we review applications of these models to psychiatric questions, identify methodological challenges, and highlight trends of convergence among computational neuroimaging approaches. We conclude by outlining a translational neuromodeling strategy, highlighting the importance of openly available datasets from prospective patient studies for evaluating the clinical utility of computational models.

show abstract

The Dopaminergic Midbrain Encodes the Expected Certainty about Desired Outcomes

Cited by 193 publications

References 75 publications

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals

Separate mesocortical and mesolimbic pathways encode effort and reward learning signals

Computational neuroimaging strategies for single patient predictions

Translational Perspectives for Computational Neuroimaging

Contact Info

Product

Resources

About