Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI

Colas, Jaron T.; Pauli, Wolfgang M.; Larsen, Tobias; Tyszka, J. Michael; O’Doherty, John P.

doi:10.1371/journal.pcbi.1005810

Cited by 18 publications

(68 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This stands in contrast to our finding that classifier accuracy in two striatal regions distinctly correlated with two behavioral measures of learning during Pavlovian conditioning. While we did not predict an absence of a correlation between classifier accuracy in OFC and behavioral measures, it is nevertheless interesting to speculate how this finding relates to previous findings 53,54 . One possible explanation is that while the OFC is involved in encoding the relationship between stimuli and expected reward, as well as in encoding stimulus–stimulus associations, these representations are not directly utilized to drive behavior in the OFC.…”

Section: Discussioncontrasting

confidence: 88%

Evidence for model-based encoding of Pavlovian contingencies in the human brain

et al. 2019

Self Cite

View full text Add to dashboard Cite

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning. We found that dissociable sub-regions of the striatum encode predictions of stimulus-stimulus associations and predictive value, in a manner that is directly related to learning performance. Activity patterns in the orbitofrontal cortex were also found to be related to stimulus-stimulus as well as value encoding. These results suggest that the brain encodes model-based representations during Pavlovian conditioning, and that these representations are utilized in the service of behavior.

show abstract

Section: Discussioncontrasting

confidence: 88%

Evidence for model-based encoding of Pavlovian contingencies in the human brain

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Such transfer is related to striatal correlates of learned value . While our in-task learning measures are related to ("Yes" / "No") action value, this previous work and recent human fMRI research indicate that mechanisms supporting the learning of stimulus-action and stimulus-value associations operate at the same time (Colas et al, 2017).…”

Section: Participants and Overview Participants Were Recruited Via Amentioning

confidence: 57%

Reward learning over weeks versus minutes increases the neural representation of value in the human brain

Wimmer

Gorgolewski

et al. 2017

Preprint

View full text Add to dashboard Cite

Over the past few decades, neuroscience research has illuminated the neural mechanisms supporting learning from reward feedback. Learning paradigms are increasingly being extended to study mood and psychiatric disorders as well as addiction. However, one potentially critical characteristic that this research ignores is the effect of time on learning: human feedback learning paradigms are usually conducted in a single rapidly paced session, while learning experiences in ecologically relevant circumstances and in animal research are almost always separated by longer periods of time. In our experiments, we examined reward learning distributed across weeks vs.learning completed in a traditionally-paced "massed" single session. As expected, we found that after equal amounts of training, accuracy was matched between the spaced and massed conditions. However, in a 3-week follow-up, we found that participants exhibited significantly greater memory for the value of spaced-trained stimuli.Supporting a role for short-term memory in massed learning, we found a significant positive correlation between initial learning and working memory capacity. Neurally, we found that patterns of activity in the medial temporal lobe and prefrontal cortex showed stronger discrimination of spaced-vs. massed-trained reward values. Further, patterns in the striatum discriminated between spaced-and massed-trained stimuli overall. Our results indicate that single-session learning tasks likely engage different learning mechanisms than spaced training. Our studies begin to address a large gap in our knowledge of human reinforcement learning, with potentially broad implications for our understanding of learning in mood disorders and addiction.. CC-BY-NC-ND 4.0 International license peer-reviewed) is the author/funder. It is made available under aThe copyright holder for this preprint (which was not . http://dx.doi.org/10.1101/158964 doi: bioRxiv preprint first posted online Jul. 3, 2017; Significance statementHumans and animals learn to associate predictive value with stimuli and actions, and these values then guide future behavior. Such reinforcement-based learning often happens over long time periods, in contrast to most studies of reward-based learning inhumans. In experiments that tested the effect of spacing on learning, we found that associations learned in a single massed session were associated with short-term memory capacity and significantly decayed over time, while associations learned over weeks were well-maintained. Additionally, we found that patterns of activity in the medial temporal lobe and prefrontal cortex discriminated the values of stimuli learned over weeks but not minutes. These results highlight the importance of studying learning over time, with potential applications to drug addiction and psychiatry.

show abstract

“…Another study, with the CIT168 atlas as reference, combined electrophysiological, lesion, and functional MRI data, and found evidence that the human amygdala processes both the degree of emotion in facial expressions and the categorical ambiguity of the emotion shown, and that these two aspects of amygdala processing can be most clearly distinguished at the level of single neurons [10]. The present extension of the CIT168 has already been used in a recent functional MRI study, which found evidence for state value prediction errors in the human substantia nigra (pars compacta), while participants solved a Markovian decision making task [11].…”

Section: Background and Summarymentioning

confidence: 85%

A High-Resolution Probabilistic In Vivo Atlas of Human Subcortical Brain Nuclei

Pauli

Nili²,

Tyszka

2017

Preprint

Self Cite

View full text Add to dashboard Cite

Recent advances in magnetic resonance imaging (MRI) methods, including data acquisition, pre-processing and analysis, have enabled research on the contributions of subcortical brain nuclei to human cognition and behavior. At the same time, these developments have led to an increasing need for a high-resolution probabilistic in-vivo anatomical atlas of subcortical nuclei. In order to fill this gap, we constructed high spatial resolution, three-dimensional templates, using joint high accuracy diffeomorphic registration of T1-and T2-weighted structural images from 168 typical adults between 22 and 35 years old. In these templates, many tissue boundaries are clearly visible, which would otherwise be impossible to delineate in data from individual studies. The resulting delineation provides a more accurate parcellation of subcortical nuclei than current histology-based atlases. We further created a companion library of software tools for atlas development, to offer an open and evolving resource for the creation of a crowd-sourced in-vivo probabilistic anatomical atlas of the human brain.

show abstract

Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI

Cited by 18 publications

References 75 publications

Evidence for model-based encoding of Pavlovian contingencies in the human brain

Evidence for model-based encoding of Pavlovian contingencies in the human brain

Reward learning over weeks versus minutes increases the neural representation of value in the human brain

A High-Resolution Probabilistic In Vivo Atlas of Human Subcortical Brain Nuclei

Contact Info

Product

Resources

About