Multiple associative structures created by reinforcement and incidental statistical learning mechanisms

Klein-Flügge, Miriam C.; Wittmann, Marco K.; Shpektor, Anna; Jensen, Daria E. A.; Rushworth, Matthew

doi:10.1038/s41467-019-12557-z

Cited by 38 publications

(40 citation statements)

References 80 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, the preferential encoding of actor prediction errors by the striatum and dopaminergic midbrain is consistent with earlier findings showing that the mesolimbic pathway preferentially encodes reward prediction errors in instrumental learning tasks (Garrison et al, 2013;Grogan et al, 2020;Hamid et al, 2019;O'Doherty, 2004). Conversely, the preferential encoding of spectator prediction errors by the mPFC is consistent with its more general role in statistical learning (Gilboa and Marlatte, 2017;Klein-Flügge et al, 2019). It may explain why dysfunctions of this structure impair observational learning more profoundly than instrumental learning (Jurado-Parras et al, 2012;Kumaran et al, 2015) and why mPFC lesions can alter the perception of controllability without altering performance of simple instrumental learning tasks (O'Callaghan et al, 2019).…”

Section: Discussionsupporting

confidence: 88%

Stress-sensitive inference of task controllability

Ligneul

Mainen

et al. 2020

Preprint

View full text Add to dashboard Cite

Estimating environmental controllability enables agents to better predict upcoming events and decide when to engage controlled action selection. How does the human brain estimate environmental controllability? Trial-by-trial analysis of choices, decision times and neural activity in an explore-and-predict task demonstrate that humans solve this problem by comparing the predictions of an “actor” model with those of a reduced “spectator” model of their environment. Neural BOLD responses within striatal and medial prefrontal areas tracked the instantaneous difference in the prediction errors generated by these two statistical learning models. BOLD activity in the posterior cingulate, parietal and prefrontal cortices covaried with changes in estimated controllability. Exposure to inescapable stressors biased controllability estimates downward and increased reliance on the spectator model in an anxiety-dependent fashion. Taken together, these findings provide a mechanistic account of controllability inference and its distortion by stress exposure.

show abstract

Section: Discussionsupporting

confidence: 88%

Stress-sensitive inference of task controllability

Ligneul

Mainen

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…As such, our findings suggest that the contribution of OFC to decision making may be limited to situations that require model-based planning, and that choices based on direct experience may rely on value computations in other brain areas, such as the amygdala or striatum (Paton et al, 2006;Cox and Witten, 2019). This proposal is seemingly at odds with a large number of studies across different species that has shown neural correlates of both inferred and directly experienced value in OFC (Hare et al, 2009;Schoenbaum et al, 2009;Barron et al, 2013;Stalnaker et al, 2014;Howard et al, 2015;Padoa-Schioppa and Conen, 2017;Suzuki et al, 2017;Klein-Flugge et al, 2019;Lopez-Persem et al, 2020;Wang et al, 2020). Why would OFC represent value signals if they are not required for behavior?…”

Section: Discussionmentioning

confidence: 72%

Targeted stimulation of an orbitofrontal network disrupts decisions based on inferred, not experienced outcomes

Wang

Howard

Voss

et al. 2020

Preprint

View full text Add to dashboard Cite

ABSTRACTWhen direct experience is unavailable, animals and humans can imagine or infer the future to guide decisions. Behavior based on direct experience versus inference may recruit distinct but overlapping brain circuits. In rodents, the orbitofrontal cortex (OFC) contains neural signatures of inferred outcomes, and OFC is necessary for behavior that requires inference but not for responding driven by direct experience. In humans, OFC activity is also correlated with inferred outcomes, but it is unclear whether OFC activity is required for inference-based behavior. To test this, we used non-invasive network-based continuous theta burst stimulation (cTBS) to target lateral OFC networks in the context of a sensory preconditioning task that was designed to isolate inference-based behavior from responding that can be based on direct experience alone. We show that relative to sham, cTBS targeting this network impairs reward-related behavior in conditions in which outcome expectations have to be mentally inferred. In contrast, OFC-targeted stimulation does not impair behavior that can be based on previously experienced stimulus-outcome associations. These findings suggest that activity in the targeted OFC network supports decision making when outcomes have to be mentally simulated, providing converging cross-species evidence for a critical role of OFC in model-based but not model-free control of behavior.

show abstract

“…As such, our findings suggest that the contribution of OFC to decision-making may be limited to situations that require model-based planning, and that choices based on direct experience may rely on value computations in other areas, such as the amygdala or striatum ( Paton et al, 2006 ; Cox and Witten, 2019 ). This proposal is seemingly at odds with the large number of studies across different species that consistently report neural correlates of both inferred and directly experienced value in OFC ( Hare et al, 2009 ; Schoenbaum et al, 2009 ; Barron et al, 2013 ; Stalnaker et al, 2014 ; Howard et al, 2015 ; Padoa-Schioppa and Conen, 2017 ; Suzuki et al, 2017 ; Klein-Flügge et al, 2019 ; Lopez-Persem et al, 2020 ; Wang et al, 2020 ). Why would OFC represent value signals if they are not required for behavior?…”

Section: Discussionmentioning

confidence: 98%

Targeted Stimulation of an Orbitofrontal Network Disrupts Decisions Based on Inferred, Not Experienced Outcomes

et al. 2020

View full text Add to dashboard Cite

When direct experience is unavailable, animals and humans can imagine or infer the future to guide decisions. Behavior based on direct experience versus inference may recruit partially distinct brain circuits. In rodents, the orbitofrontal cortex (OFC) contains neural signatures of inferred outcomes, and OFC is necessary for behavior that requires inference but not for responding driven by direct experience. In humans, OFC activity is also correlated with inferred outcomes, but it is unclear whether OFC activity is required for inference-based behavior. To test this, we used noninvasive network-based continuous theta burst stimulation (cTBS) in human subjects (male and female) to target lateral OFC networks in the context of a sensory preconditioning task that was designed to isolate inference-based behavior from responding that can be based on direct experience alone. We show that, relative to sham, cTBS targeting this network impairs reward-related behavior in conditions in which outcome expectations have to be mentally inferred. In contrast, OFC-targeted stimulation does not impair behavior that can be based on previously experienced stimulus–outcome associations. These findings suggest that activity in the targeted OFC network supports decision-making when outcomes have to be mentally simulated, providing converging cross-species evidence for a critical role of OFC in model-based but not model-free control of behavior. SIGNIFICANCE STATEMENT It is widely accepted that the orbitofrontal cortex (OFC) is important for decision-making. However, it is less clear how exactly this region contributes to behavior. Here we test the hypothesis that the human OFC is only required for decision-making when future outcomes have to be mentally simulated, but not when direct experience with stimulus–outcome associations is available. We show that targeting OFC network activity in humans using network-based continuous theta burst stimulation selectively impairs behavior that requires inference but does not affect responding that can be based solely on direct experience. These results are in line with previous findings in animals and suggest a critical role for human OFC in model-based but not model-free behavior.

show abstract

Multiple associative structures created by reinforcement and incidental statistical learning mechanisms

Cited by 38 publications

References 80 publications

Stress-sensitive inference of task controllability

Stress-sensitive inference of task controllability

Targeted stimulation of an orbitofrontal network disrupts decisions based on inferred, not experienced outcomes

Targeted Stimulation of an Orbitofrontal Network Disrupts Decisions Based on Inferred, Not Experienced Outcomes

Contact Info

Product

Resources

About