Ventromedial Prefrontal Cortex Encodes a Latent Estimate of Cumulative Reward

Juechems, Keno; Balaguer, Jan; Ruz, Marı́a; Summerfield, Christopher

doi:10.1016/j.neuron.2016.12.038

Cited by 42 publications

(89 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These findings build on our previous work implicating the human vmPFC in coding an internal, unsignalled representation of cumulative assets in a way that maximises a specific goal (maintaining net positive aggregate reward), over and above any momentary hedonic value that arises from choices 23 . However, the current work additionally implies why the brain keeps track of cumulative assets – because this allows any imbalance among internal needs to be redressed by future choices.…”

Section: Discussionsupporting

confidence: 76%

“…bandit or food choice tasks) that involve maximisation of a single asset, rather than those that require the balancing of multiple assets. Here, we built on a recent finding that medial OFC tracks the internal state given by cumulative satiety or wealth over a sequence of trials 23 , 24 . We reasoned that by encoding these signals, the OFC and interconnected areas of the medial prefrontal cortex might allow animals to compute ongoing levels of competing assets, and dynamically adjust choices to both replenish immediate needs and guard against future scarcity.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A network for computing value homeostasis in the human medial prefrontal cortex

Juechems

Balaguer

Castañón

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

20Humans and other animals make decisions in order to satisfy their goals. However, it remains 21 unknown how neural circuits compute which of multiple possible goals should be pursued (e.g. 22 when balancing hunger and thirst) and combine these signals with estimates of available reward 23 alternatives. Here, humans undergoing functional magnetic resonance imaging (fMRI) 24 accumulated two distinct assets over a sequence of trials. Financial outcomes depended on the 25 minimum cumulate of either asset, creating a need to maintain "value homeostasis" by 26 redressing any imbalance among the assets. BOLD signals in the dorsal anterior cingulate 27 cortex (dACC) tracked the level of homeostatic imbalance among goals, whereas the 28 ventromedial prefrontal cortex (vmPFC) signalled the level of homeostatic redress incurred by 29 a choice, rather than the overall amount received. These results suggest that a network of medial 30 frontal brain regions compute a value signal that maintains homeostatic balance among internal 31 goals. 32 33 3Canonical models in psychology, economics, and machine learning assume that decisions are 34 made in order to maximise expected reward. One popular view proposes that estimates of 35 reward for distinct stimuli or attributes (e.g. price and quality) are collapsed into a common 36 value signal that motivates reward-based decisions 1-3 . In humans, BOLD signals 4-6 in the 37 medial orbitofrontal cortex (OFC) and ventromedial prefrontal cortex code for food items 7,8 , 38 money 9,10 and social stimuli 11 , suggesting the existence of a domain-general value code in this 39 region. In non-human primates and rodents, neuronal firing rates 12-14 in lateral and central 40 orbitofrontal cortex (OFC) encode value signals for primary reinforcers. Moreover, humans 41 and primates with medial OFC lesions fail to integrate multiple attributes of a stimulus when 42 making decisions, implying that the OFC plays an active role in constructing composite value 43 estimates for available stimuli [15][16][17][18] . 44 However, in natural environments, survival depends on the minimum level of any one of a 45 number of competing internal needs. For example, a hungry animal needs food more than 46 water, whereas the reverse is true for a thirsty animal. As the world changes dynamically, 47 distinct internal assets (e.g. satiety or hydration) are continuously being depleted and 48 replenished, so that homeostasis among internal resource levels needs to be monitored and 49 maintained in neural circuits for valuation and choice [19][20][21] . The OFC is likely to play a role in 50 evaluating stimuli in the context of internal goal states (e.g. satisfy hunger or quench thirst), 51 because animals with OFC lesions continue to choose actions that lead to devalued outcomes, 52 as if they were failing to maintain or follow currently active goals 18,22 . However, a key open 53 question is how the brain evaluates which of multiple possible goals should be pursued at any 54 one time, and how it dynamically upda...

show abstract

Section: Discussionsupporting

confidence: 76%

mentioning

confidence: 99%

A network for computing value homeostasis in the human medial prefrontal cortex

Juechems

Balaguer

Castañón

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Loss aversion has been subject to challenges in recent years from research that emphasizes context dependence. These studies have found consistent shifts in decisionmaking as a function of recent history (Jeuchems, Balaguer, Ruz, & Summerfield, 2017;Post, van den Assem, Baltussen, & Thaler, 2008;Rigoli et al, 2016) or current context (Louie, Khaw, & Glimcher, 2013;Yamada, Louie, Tymula, & Glimcher, 2018) explained by biologically plausible normalization models that consider values relative to their context.…”

Section: Challenges To Loss Aversionmentioning

confidence: 84%

The Psychological and Neural Basis of Loss Aversion

Sokol‐Hessner

Rutledge

2018

Curr Dir Psychol Sci

105

View full text Add to dashboard Cite

Loss aversion is a central element of Prospect Theory, the dominant theory of decisionmaking under uncertainty for the past four decades, and refers to the overweighting of potential losses relative to equivalent gains, a critical determinant of risky decision-making. Recent advances in affective and decision neuroscience have shed new light on the psychological and neurobiological mechanisms underlying loss aversion. Here, integrating across disparate literatures, from the level of neurotransmitters to subjective reports of emotion, we propose a novel neural and computational framework that links norepinephrine to loss aversion and identifies a distinct role for dopamine in risk taking for rewards. We also propose that loss aversion specifically relates to anticipated emotions and aspects of the immediate experience of realized gains and losses but not their longterm emotional consequences, highlighting an underappreciated temporal structure. Finally, we discuss challenges to loss aversion and the relevance of loss aversion to understanding psychiatric disorders. Refining models of loss aversion will have broad consequences for the science of decision-making, and for how we understand individual variation in economic preferences and psychological well-being across both healthy and psychiatric populations.

show abstract

“…The observation that vmPFC activity is sensitive to different reward types (e.g., money, food), across a variety of choice tasks, has led to the proposal that vmPFC represents a "common currency" of subjective value (Levy & Glimcher, 2012). Recent fMRI evidence suggests that vmPFC not only responds to momentary gains and losses, but also tracks cumulative reward histories (Juechems, Balaguer, Ruz, & Summerfield, 2017) that are accompanied by shifts in risk preference. Support for the idea that vmPFC plays a role in the construction of subjective utility has also come from lesion studies in which patients with focal vmPFC damage were compared to patients with damage to other brain regions, as well as to healthy controls.…”

Section: Value and The Vmpfcmentioning

confidence: 99%

Differential impact of ventromedial prefrontal cortex damage on “hot” and “cold” decisions under risk

Spaniol

Muro

Ciaramelli

2018

Cogn Affect Behav Neurosci

View full text Add to dashboard Cite

The ventromedial prefrontal cortex (vmPFC) is known to play a key role in reward processing and decision making. However, its relative contribution to affect-rich (Bhot^) and affect-poor (Bcold^) decisions is not fully understood. Damage to vmPFC is associated with impaired performance on laboratory tasks of decision making under ambiguity and risk. In the current study, we tested the hypothesis that vmPFC is critical for adaptive risk taking under Bhot^conditions specifically. Participants included patients with focal lesions in vmPFC, patient controls with damage in regions not including vmPFC, and healthy controls. They completed hot and cold versions of a dynamic risk-taking task, the Columbia Card Task (CCT). Relative to healthy controls and patient controls, vmPFC patients showed a strong overall increase in risk taking in the hot version of the CCT, despite preserved sensitivity to trial-level variation in risk. In the cold version, overall risk taking was similar among all three groups, even though vmPFC patients showed reduced sensitivity to trial-level variation in risk. Sensitivity to gain and loss magnitudes did not differ significantly among the groups, in either the hot or the cold CCT. These findings lend novel support to the hypothesis that the vmPFC is critical for adaptive decision making under affect-rich conditions.

show abstract

Ventromedial Prefrontal Cortex Encodes a Latent Estimate of Cumulative Reward

Cited by 42 publications

References 71 publications

A network for computing value homeostasis in the human medial prefrontal cortex

A network for computing value homeostasis in the human medial prefrontal cortex

The Psychological and Neural Basis of Loss Aversion

Differential impact of ventromedial prefrontal cortex damage on “hot” and “cold” decisions under risk

Contact Info

Product

Resources

About