Real-Time Value Integration during Economic Choice Is Regulated by Orbitofrontal Cortex

Gardner, Michael O.; Conroy, Jessica C.; Sanchez, Davied C.; Zhou, Jingfeng; Schoenbaum, Geoffrey

doi:10.1016/j.cub.2019.10.058

Cited by 30 publications

(31 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One possible resolution to this tension is a difference between species (either biological or in terms of experimental implementations). In rats, silencing OFC does not cause impairments on an economic choice task 21,22 . Another possible resolution is a difference between tasks.…”

Section: Discussionmentioning

confidence: 91%

“…Lesion studies in primates have suggested that different subregions of OFC may selectively support learning and choosing [13][14][15] . In rodents, considerable evidence suggests that disrupting neural activity in the OFC impairs learning [16][17][18][19][20] , but it remains controversial whether this also impairs choice [21][22][23] . Recording studies in many species have revealed neural correlates of expected value in the OFC [24][25][26][27][28] , but it remains unclear whether these representations are selective for roles in learning and in choosing.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Value Representations in the Rodent Orbitofrontal Cortex Drive Learning, not Choice

Miller

Botvinick

Brody

2018

Preprint

View full text Add to dashboard Cite

5 Co-corresponding authors As humans and animals experience the world, they learn to associate states and actions with the expected values of the reward that is likely to follow [1][2][3] . Neural correlates of expected value are found in many brain regions, including the orbitofrontal cortex (OFC) [4][5][6][7][8][9] . While OFC value representations have been identified across many tasks and species [10][11][12][13][14][15] , their computational role remains controversial [16][17][18] . One influential hypothesis holds that they drive value-based choosing: The OFC represents the expected values of available options, and choices are made by comparing these values to one another 4,7,9 . A contrasting hypothesis holds that they drive learning: The OFC represents the expected values of immediately impending outcomes, which are compared to rewards actually received, so as to learn and adapt expectations to match the world 5,6,19,20 . In common laboratory tasks the items to be decided between are also the items to be learned about, making the two hypothesized roles difficult to distinguish. Here, we use a recently-developed multi-step task for rats 21 that separates choosing from learning. In a first step, rats choose one of two ports ("choice ports") whose expected values are computed using planning, and are not learned. In the second step, rats are led to one of two other ports ("outcome ports") which are not chosen between, but whose expected values are learned based on reward history. We found relatively weak OFC encoding of choice port values, needed for choosing but not learning, but far stronger encoding of outcome port values, needed for learning but not choosing. Moreover, temporally-specific silencing of OFC during outcome port entry was sufficient to disrupt behavior, and the nature of this disruption was consistent with impairment of a value learning process, but was not consistent with impairment of a choice process. We therefore suggest that value representations in the OFC directly drive learning, but do not directly drive choice.We trained rats on a two-step decision task, adapted from the human literature 22 , in which a choice made by the subject in a first step is probabilistically, not deterministically, linked to an outcome that occurs in a second step ( Fig, 1a ). In each trial of our rat version of this task 21 , the rat first initiated the trial by poking its nose into a neutral center port, and then made a decision between one of two choice ports ( Fig. 1a i,ii ). One choice caused a left outcome port to become available with high probability ("common" transition), and a right outcome port to become available with low probability ("uncommon" transition), while the opposite choice reversed these probabilities ( Fig. 1a iii) . Following the initial choice, an auditory tone informed the rat which of 1 . CC-BY-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The cop...

show abstract

Section: Discussionmentioning

confidence: 91%

Section: Introductionmentioning

confidence: 99%

Value Representations in the Rodent Orbitofrontal Cortex Drive Learning, not Choice

Miller

Botvinick

Brody

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…We targeted reward-related cortical and subcortical structures of non-human primates (Haber & Knutson, 2010) including the central orbitofrontal cortex (cOFC, area 13M), the medial orbitofrontal cortex, (mOFC, area 14O), the dorsal striatum (DS, caudate nucleus), and ventral striatum (VS), all of which are known to represent the neural correlate with stimulus values during economic choice behavior. To dissociate the integrative process to compute expected values from a choice process employed during economic choices (Chen & Stuphorn, 2015; Gardner et al, 2019; Yoo & Hayden, 2020), we recorded the neural activity in a non-choice situation; monkeys perceive and compute the expected values from probability and magnitude symbols. We used a recently developing mathematical approach, called state space analysis (Chen & Stuphorn, 2015; Churchland et al, 2012; Mante et al, 2013; Murray et al, 2017), to test how the expected-values computation is processed within each of the four neural population ensembles in an order of 10 −2 -second time resolution.…”

Section: Introductionmentioning

confidence: 99%

Neural population dynamics underlying expected value computation

Yamada

Imaizumi

Matsumoto

2020

Preprint

View full text Add to dashboard Cite

21Computation of expected values, i.e., probability times magnitude, seems to be a 22 dynamic integrative process performed in the brain for efficient economic behavior. 23 However, neural dynamics underlying this computation remain largely unknown. 24 We examined (1) whether four core reward-related regions detect and integrate 25 the probability and magnitude cued by numerical symbols and (2) whether these 26 regions have different dynamics in the integrative process. Extractions of 27 mechanistic structure of neural population signal demonstrated that expected-28 value signals simultaneously arose in central part of orbitofrontal cortex (cOFC, 29 area 13m) and ventral striatum (VS). These expected-value signals were incredibly 30 stable in contrast to weak and unstable signals in dorsal striatum and medial OFC. 31Notably, temporal dynamics of these stable expected-value signals were 32 unambiguously distinct: sharp and gradual signal evolutions in cOFC and VS, 33 respectively. These intimate dynamics suggest that cOFC and VS compute the 34 expected-values with unique time constants, as distinct, partially overlapping 35 processes. 36 37 Impact Statement 38 Our empirical study on neural population dynamics suggests that the orbitofrontal 39 cortex and ventral striatum co-operate on expected-value computation with unique 40 time constants, as distinct, partially overlapping processes.41 42 107 1 where 1 indicates a 100% chance). After a 2.5 second delay, the visual pie-chart 108 disappeared, and a reward outcome was provided to the monkeys with the 109 indicated amount and probability of reward, unless no reward was given. Under110 6 this experimental condition, the expected values of rewards are defined as the 111 probability multiplied by the magnitude cued by the numerical symbols (Table. S1). 112To examine whether the monkeys accurately perceived the expected values 113 from the numerical symbols for probability and magnitude, we applied a choice 114 task to the monkeys (Fig. 1B). The two monkeys exhibited a near-efficient 115 performance in selecting a larger expected value option among two alternatives 116 during choice trials (Fig. 1C). We examined which of the following three behavioral 117 models best described the monkey's behavior: model 1 (M1), monkeys make 118 choices based on the number of pies; model 2 (M2), monkeys make choices based 119 on the probability and magnitude; and model 3 (M3), monkeys make choices based 120 on the expected values (Table. S1). Comparisons of the model performances 121 based on Akaike's Information Criterion (AIC) and Bayesian Information Criterion 122 (BIC) (Burnham & Anderson, 2004) revealed that the model 3 best explained the 123 monkey's behavior as indicated by the smallest AIC and BIC values (Fig. S1). The 124 model 3 consistently showed the highest pseudo r-squared values in each monkey 125 (Fig. 1D). These results indicated that the monkeys utilize the expected values 126 estimated from the numerical symbols for probability and magnitude. 127 128Neural pop...

show abstract

“…In our study of the neurobiology of fear reduction, we also chose to move beyond the traditional fear circuit. Our candidate was the lateral orbitofrontal cortex (lOFC), a structure strongly linked to reward learning (e.g., Gardner et al, 2019 ; Padoa-Schioppa and Assad, 2006 ; Rich and Wallis, 2016 ). We targeted the OFC for three reasons.…”

Section: Introductionmentioning

confidence: 99%

Different methods of fear reduction are supported by distinct cortical substrates

Lay

Pitaru

Boulianne

et al. 2020

eLife

View full text Add to dashboard Cite

Understanding how learned fear can be reduced is at the heart of treatments for anxiety disorders. Tremendous progress has been made in this regard through extinction training in which the aversive outcome is omitted. However, current progress almost entirely rests on this single paradigm, resulting in a very specialized knowledgebase at the behavioural and neural level of analysis. Here, we used a dual-paradigm approach to show that different methods that lead to reduction in learned fear in rats are dissociated in the cortex. We report that the infralimbic cortex has a very specific role in fear reduction that depends on the omission of aversive events but not on overexpectation. The orbitofrontal cortex, a structure generally overlooked in fear, is critical for downregulating fear when novel predictions about upcoming aversive events are generated, such as when fear is inflated or overexpected, but less so when an expected aversive event is omitted.

show abstract

Real-Time Value Integration during Economic Choice Is Regulated by Orbitofrontal Cortex

Abstract: Highlights d Rats show immediate changes in choice behavior following reinforcer revaluation d Direction of satiety-specific revaluation depends on the baseline food preference d Orbitofrontal inactivation disrupts behavior following reinforcer revaluation

Cited by 30 publications

References 46 publications

Value Representations in the Rodent Orbitofrontal Cortex Drive Learning, not Choice

Value Representations in the Rodent Orbitofrontal Cortex Drive Learning, not Choice

Neural population dynamics underlying expected value computation

Different methods of fear reduction are supported by distinct cortical substrates

Contact Info

Product

Resources

About