Dopamine Enhances Model-Free Credit Assignment Through Boosting of Retrospective Model-Based Inference

Deserno, Lorenz; Moran, Rani; Lee, Ying; Michely, Jochen; Dayan, Peter; Dolan, Raymond J.

doi:10.1016/j.biopsych.2021.02.246

Cited by 4 publications

(2 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Still, previous work indicated that such distinction might not be appropriate by showing that the MF system can have access to MB information (e.g. Daw et al, 2011;Deserno et al, 2021;Moran et al, 2019). Second, the association might arise from inherent experimental features given that our model-based index and the choice signature of feedback learning were derived from two highly interdependent tasks.…”

Section: Discussionmentioning

confidence: 94%

Cortical grey matter mediates increases in model-based control and learning from positive feedback from adolescence to adulthood

Scholz

Waltmann

Herzog

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Adolescents undergo maturation in cognition and brain structure. Model-based (MB) control is known to increase from childhood to young adulthood, which is mediated by cognitive abilities. Here, we asked two questions unaddressed in previous developmental studies: Firstly, what are the brain structural correlates of age-related increases in MB control? Secondly, how are age-related increases in MB control from adolescence to adulthood influenced by motivational context? A developmental sample (n=103, age: 12-42) completed structural MRI and an established task to capture MB control. The task was modified with respect to outcome valence by including (1) reward and punishment blocks to manipulate the motivational context and (2) an additional choice test to assess learning from positive vs. negative feedback. After replicating that an age-dependent increase in MB control is mediated by cognitive abilities, we demonstrate first-time evidence that grey matter density (GMD) in the parietal cortex mediates the increase of MB control with age. While motivational context did not relate to age-related changes in MB control, learning from positive feedback improved with age. Meanwhile, negative feedback learning showed no age effects. We present a first report that an age-related increase in learning from positive feedback was mediated by reduced GMD in the parietal, medial and dorsolateral prefrontal cortex. Our findings indicate that efficient brain maturation, as putatively reflected in lower GMD, in distinct and partially overlapping brain regions is a key developmental step towards age-related increases in planning and value-based choice.

show abstract

Section: Discussionmentioning

confidence: 94%

Cortical grey matter mediates increases in model-based control and learning from positive feedback from adolescence to adulthood

Scholz

Waltmann

Herzog

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…People can become more model-based if we increase the incentive by increasing the amount of available payoff (Patzelt et al, 2019). Also model-based behavior can be manipulated through drug administration in Wunderlich et al (2012); Deserno et al (2021). It is questionable whether increasing model-based behavior is possible via a more self-dependent method without extra financial costs and without drugs.…”

Section: Introductionmentioning

confidence: 99%

The effects of directed and free self-monitoring on goal-directed and habitual decision-making

Miandari Hossein,

Ershadmanesh,

Vahabie

et al. 2023

Preprint

View full text Add to dashboard Cite

Many studies on humans and animals have provided evidence for the contribution of goal-directed and habitual valuation systems in learning and decision-making. However, how the arbitration between these two systems is affected by other cognitive processes is not well known. Here, we study the effects of directed and free self-monitoring of one’s decisions on this arbitration. In our experiments, in a within-subject design, the subjects participated in a control and a two modified versions of the Two-step decision-making task, where we could measure each system’s contribution to decisions. We had two modified tasks. In both, every few trials subjects had to think about what they have experienced in the past trials in one of the two days. In one task, they had to designate which action was better and then report their confidence about this decision (directed self-monitoring task). In the other modification of task, they had to explain what had happened in the past few trials by talking (free self-monitoring task). We hypnotized that in both modified tasks, the behavior of the participants would shift toward goal-directed behavior because they need to think more about the structure of the task. Our experimental results showed that subjects indeed became more goal-directed in the directed self-monitoring task, but in the free self-monitoring task, they became more habitual. We would discuss the underlying reasons for these shifts in the behavior.

show abstract

Assigning the right credit to the wrong action: compulsivity in the general population is associated with augmented outcome-irrelevant value-based learning

et al. 2021

Self Cite

View full text Add to dashboard Cite

Compulsive behavior is enacted under a belief that a specific act controls the likelihood of an undesired future event. Compulsive behaviors are widespread in the general population despite having no causal relationship with events they aspire to influence. In the current study, we tested whether there is an increased tendency to assign value to aspects of a task that do not predict an outcome (i.e., outcome-irrelevant learning) among individuals with compulsive tendencies. We studied 514 healthy individuals who completed self-report compulsivity, anxiety, depression, and schizotypal measurements, and a well-established reinforcement-learning task (i.e., the two-step task). As expected, we found a positive relationship between compulsivity and outcome-irrelevant learning. Specifically, individuals who reported having stronger compulsive tendencies (e.g., washing, checking, grooming) also tended to assign value to response keys and stimuli locations that did not predict an outcome. Controlling for overall goal-directed abilities and the co-occurrence of anxious, depressive, or schizotypal tendencies did not impact these associations. These findings indicate that outcome-irrelevant learning processes may contribute to the expression of compulsivity in a general population setting. We highlight the need for future research on the formation of non-veridical action−outcome associations as a factor related to the occurrence and maintenance of compulsive behavior.

show abstract

Dopamine Enhances Model-Free Credit Assignment Through Boosting of Retrospective Model-Based Inference

Cited by 4 publications

References 59 publications

Cortical grey matter mediates increases in model-based control and learning from positive feedback from adolescence to adulthood

Cortical grey matter mediates increases in model-based control and learning from positive feedback from adolescence to adulthood

The effects of directed and free self-monitoring on goal-directed and habitual decision-making

Assigning the right credit to the wrong action: compulsivity in the general population is associated with augmented outcome-irrelevant value-based learning

Contact Info

Product

Resources

About