Unconscious reinforcement learning of hidden brain states supported by confidence

Cortese, Aurelio; Lau, Hakwan; Kawato, Mitsuo

doi:10.1038/s41467-020-17828-8

Cited by 37 publications

(38 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Participant confidence in having performed the task well was also significantly correlated with the degree of abstraction (N = 31, robust regression, slope = 0.026, t 29 = 2.69, p = 0.012, Figure 3C , bottom). In addition to the finding that confidence related to learning speed ( Figure 1E ), these results raise intriguing questions about the function of metacognition, as participants appeared to comprehend their own ability to construct and use abstractions ( Cortese et al, 2020 ).…”

Section: Resultsmentioning

confidence: 82%

Value signals guide abstraction during learning

Cortese

Yamamoto

Hashemzadeh

et al. 2021

eLife

Self Cite

View full text Add to dashboard Cite

The human brain excels at constructing and using abstractions, such as rules, or concepts. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon the valuation of sensory features. Human volunteers learned novel association rules based on simple visual features. Reinforcement-learning algorithms revealed that, with learning, high-value abstract representations increasingly guided participant behaviour, resulting in better choices and higher subjective confidence. We also found that the brain area computing value signals - the ventromedial prefrontal cortex - prioritized and selected latent task elements during abstraction, both locally and through its connection to the visual cortex. Such a coding scheme predicts a causal role for valuation. Hence, in a second experiment, we used multivoxel neural reinforcement to test for the causality of feature valuation in the sensory cortex, as a mechanism of abstraction. Tagging the neural representation of a task feature with rewards evoked abstraction-based decisions. Together, these findings provide a novel interpretation of value as a goal-dependent, key factor in forging abstract representations.

show abstract

Section: Resultsmentioning

confidence: 82%

Value signals guide abstraction during learning

Cortese

Yamamoto

Hashemzadeh

et al. 2021

eLife

Self Cite

View full text Add to dashboard Cite

show abstract

“…The findings are based on three independent data sets with variation in the experimental design and the selected target regions, and hence suggest generalizability of the findings. (Cortese, Lau, & Kawato, 2020). On the behavioral level, a meta-analysis examining psychological factors for neurofeedback efficacy has highlighted the influence of attention, motivation, and mood on training outcome (Kadosh & Staunton, 2019).…”

Section: Discussionmentioning

confidence: 99%

Putamen volume predicts real‐time fMRI neurofeedback learning success across paradigms and neurofeedback target regions

Zhao

Yao

Zweerings

et al. 2021

Human Brain Mapping

View full text Add to dashboard Cite

Real‐time fMRI guided neurofeedback training has gained increasing interest as a noninvasive brain regulation technique with the potential to modulate functional brain alterations in therapeutic contexts. Individual variations in learning success and treatment response have been observed, yet the neural substrates underlying the learning of self‐regulation remain unclear. Against this background, we explored potential brain structural predictors for learning success with pooled data from three real‐time fMRI data sets. Our analysis revealed that gray matter volume of the right putamen could predict neurofeedback learning success across the three data sets (n = 66 in total). Importantly, the original studies employed different neurofeedback paradigms during which different brain regions were trained pointing to a general association with learning success independent of specific aspects of the experimental design. Given the role of the putamen in associative learning this finding may reflect an important role of instrumental learning processes and brain structural variations in associated brain regions for successful acquisition of fMRI neurofeedback‐guided self‐regulation.

show abstract

“…Participants' reported sense of confidence in having performed the task well also significantly correlated with the degree of abstraction (N = 31, robust regression, slope = 0.026, t29 = 2.69, p = 0.012, Figure 3C, bottom). Together with confidence self-reports being predictive of learning speed ( Figure 1E), these results raise intriguing questions on the function of metacognition, as participants appeared to grasp their own ability to construct and use abstractions (40).…”

Section: Behaviour Shifts From Feature-to Abstract-based Reinforcemenmentioning

confidence: 75%

Value Shapes Abstraction During Learning

Cortese

Yamamoto

Hashemzadeh

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Abstractions are critical for flexible behaviours and efficient learning. However, how the brain forgoes the sensory dimension to forge abstract entities remains elusive. Here, in two fMRI experiments, we demonstrate a mechanism of abstraction built upon valuation of task-relevant sensory features. Human volunteers learned hidden association rules between visual features. Computational modelling of participants' choice data with mixture-of-experts reinforcement learning algorithms revealed that, with learning, emerging high-value abstract representations increasingly guided behaviour. Moreover, the brain area encoding value signals - the ventromedial prefrontal cortex - also prioritized and selected latent task elements, both locally and through its connection to visual cortex. In a second experiment, we used multivoxel neural reinforcement to show how reward-tagging the neural sensory representation of a task's feature evoked abstraction-based decisions. Our findings redefine the logic of valuation as a goal-dependent, key factor in constructing the abstract representations that govern intelligent behaviour.

show abstract

Unconscious reinforcement learning of hidden brain states supported by confidence

Cited by 37 publications

References 71 publications

Value signals guide abstraction during learning

Value signals guide abstraction during learning

Putamen volume predicts real‐time fMRI neurofeedback learning success across paradigms and neurofeedback target regions

Value Shapes Abstraction During Learning

Contact Info

Product

Resources

About