Reinforcement-learning in fronto-striatal circuits

Averbeck, Bruno B.; O’Doherty, John P.

doi:10.1038/s41386-021-01108-0

Cited by 85 publications

(79 citation statements)

References 181 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given that animal and human studies indicate that reinforcement learning is critically mediated by the functional communication between the VS and frontal regions (42,43), treatment effects on frontal-VS functional connectivity during transfer phase were examined.…”

Section: Functional Connectivity Analysismentioning

confidence: 99%

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Zhou

Kanen

et al. 2022

Preprint

View full text Add to dashboard Cite

Adaptive human learning utilizes reward prediction errors (RPEs) that scale the differences between expected and actual outcomes to optimize future choices. Depression has been linked with biased RPE signaling and an exaggerated impact of negative outcomes on learning which may promote amotivation and anhedonia. The present proof-of-concept study combined computational modelling and multivariate decoding with neuroimaging to determine the influence of the angiotensin II type 1 receptor antagonist losartan on learning from positive or negative outcomes and the underlying neural mechanisms in healthy humans. In a double-blind, between-subjects, placebo-controlled pharmaco-fMRI experiment, 61 healthy male participants (losartan, n=30; placebo, n=31) underwent a probabilistic selection reinforcement learning task incorporating a learning and transfer phase. Losartan improved choice accuracy for the hardest stimulus pair relative to the placebo group during learning. Computational modelling revealed that losartan reduced the learning rate for negative outcomes and increased exploitatory choice behaviors while preserving learning for positive outcomes. These behavioral patterns were paralleled on the neural level by increased RPE signaling in orbitofrontal-striatal regions and enhanced positive outcome representations in the ventral striatum (VS) following losartan. In the transfer phase, losartan accelerated response times and enhanced VS functional connectivity with left dorsolateral prefrontal cortex when approaching maximum rewards. These findings elucidate the potential of losartan to reduce the impact of negative outcomes during learning and subsequently facilitate motivational approach towards maximum rewards in the transfer of learning. The mechanism could help to normalize biased reward learning characteristic of depression.

show abstract

Section: Functional Connectivity Analysismentioning

confidence: 99%

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Zhou

Kanen

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…We therefore expect these regions to be involved in attention modulation in AARM, given the mechanism's critical reliance on activation of past exemplars (Point 3 in Figure 1). The midbrain dopaminergic systems and basal ganglia have been implicated in policy-updating behaviors in RL that seek to maximize reward and minimize errors (Averbeck & O'Doherty, 2021). Because attention updating in AARM is calculated according to a feedback-based error-gradient, we expect model-predicted attention to require the influence of prediction error action selection functions in these regions (Point 4 in Figure 1).…”

Section: Hypothesized Neural Systemsmentioning

confidence: 99%

“…As part of the midbrain dopaminergic system (Schultz & Romo, 1992), their role in action selection is critically influenced by reward-related influxes in dopamine (Schultz et al, 1993;Seymour et al, 2007). The striatum, for example, has been shown to be involved in RL, whereby human and animal observers intuitively learn to behave in a manner that maximizes reward and avoids harm (for review, see Averbeck & O'Doherty, 2021).…”

Section: Midbrain Dopaminergic Systems and The Basal Gangliamentioning

confidence: 99%

“…A compelling line of computational and neurophysiology work (Humphries et al, 2012;Frank et al, 2004Frank et al, , 2007Frank et al, , 2009 has suggested that the explore vs. exploit tradeoff is directly modulated by striatal dopamine, such that increasing tonic striatal dopamine decreases the probability of explorative action selection output from the basal ganglia to the superior colliculus. fMRI studies have additionally shown that exploration tends to engage the frontal pole whereas exploitation engages the vmPFC (Daw et al, 2006), suggesting dissociable downstream executive effects of action selection via the basal ganglia (Averbeck & O'Doherty, 2021).…”

Section: Midbrain Dopaminergic Systems and The Basal Gangliamentioning

confidence: 99%

See 1 more Smart Citation

Distributed neural systems enable flexible attention updating during category learning

Weichart¹,

Evans²,

Galdo³

et al. 2021

Preprint

View full text Add to dashboard Cite

In order to accurately categorize novel items, humans learn to selectively attend to stimulus dimensions that are most relevant to the task. Models of category learning describe the interconnected cognitive processes that contribute to selective attention as observations of stimuli and category feedback are progressively acquired. The Adaptive Attention Representation Model (AARM), for example, provides an account whereby categorization decisions are based on the perceptual similarity of a new stimulus to stored exemplars, and dimension-wise attention is updated on every trial in the direction of a feedback-based error gradient. As such, attention modulation as described by AARM requires interactions among orienting, visual perception, memory retrieval, error monitoring, and goal maintenance in order to facilitate learning across trials. The current study explored the neural bases of attention mechanisms using quantitative predictions from AARM to analyze behavioral and fMRI data collected while participants learned novel categories. GLM analyses revealed patterns of BOLD activation in the parietal cortex (orienting), visual cortex (perception), medial temporal lobe (memory retrieval), basal ganglia (error monitoring), and prefrontal cortex (goal maintenance) that covaried with the magnitude of model-predicted attentional tuning. Results are consistent with AARM’s specification of attention modulation as a dynamic property of distributed cognitive systems.

show abstract

“…Studies of RL usually rely on n-armed bandit tasks (Averbeck and O'Doherty, 2021). In these tasks, participants are shown (B) Each session included i sets, i = 32 for acute recordings, i = 108 for chronic recordings; each set had j trials, j˛½10;30.…”

Section: Introductionmentioning

confidence: 99%

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

Reinforcement-learning in fronto-striatal circuits

Cited by 85 publications

References 181 publications

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Angiotensin blockade enhances motivational reward learning via enhancing striatal prediction error signaling and frontostriatal communication

Distributed neural systems enable flexible attention updating during category learning

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior

Contact Info

Product

Resources

About