Reinforcement Learning and Gilles de la Tourette Syndrome

Worbe, Yulia; Palminteri, Stefano; Hartmann, Andréas; Vidailhet, Marie; Lehericy, Stéphane; Pessiglione, Mathias

doi:10.1001/archgenpsychiatry.2011.137

Cited by 49 publications

(31 citation statements)

References 76 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Dopaminergic manipulations during choice, in ways that could not have affected learning, show that increasing dopamine increases the weighting of positive relative to negative values, confirming an effect on performance (43,44). Dopaminergic manipulations during learning modulate PE signaling (45–48), in ways that are predictive of subsequent choice (45–47), confirming an effect on learning. Consistent with these dual effects, optogenetic stimulation and inhibition of dopamine neurons cause appetitive and aversive learning, respectively, if done at outcome, but increase and decrease approach behavior, respectively, if done during choice (36).…”

Section: Computational Roles Of Dopaminementioning

confidence: 72%

An Integrative Perspective on the Role of Dopamine in Schizophrenia

Maia

Frank

2017

Biological Psychiatry

252

233

View full text Add to dashboard Cite

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing. Increased spontaneous phasic dopamine release helps to explain positive symptoms and provides a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with positive symptoms: aberrant learning for neutral cues (assessed with behavioral and autonomic responses), and aberrant, increased activation of the ventral striatum, midbrain, and other limbic regions for neutral cues, neutral outcomes, and neutral prediction errors. Taken together, then, these two disturbances explain many findings in schizophrenia. We review evidence supporting their co-occurrence and consider their differential implications for the treatment of positive and negative symptoms.

show abstract

Section: Computational Roles Of Dopaminementioning

confidence: 72%

An Integrative Perspective on the Role of Dopamine in Schizophrenia

Maia

Frank

2017

Biological Psychiatry

252

233

View full text Add to dashboard Cite

show abstract

“…For instance, reward-seeking behaviour and reward prediction errors encoded in the ventral striatum were reduced by haloperidol administration in healthy volunteers 11 . In contrast, both functions were preserved in GTS patients medicated with aripiprazole 12 . These results are consistent with the idea that, as opposed to dopamine receptor antagonists, aripiprazole preserves motivational functions via preserving reward-related signals.…”

Section: Introductionmentioning

confidence: 77%

“…GTS patients represented an ideal model for this investigation because in this population we have already shown the beneficial effects of aripiprazole compared to a dopamine receptor antagonist in factual learning, that is, in learning from obtained outcomes (as opposed to counterfactual learning, which is learning from forgone outcomes) 12 . In addition, unlike many schizophrenic patients, GTS patients have relatively preserved cognitive efficiency 23, 24 .…”

Section: Introductionmentioning

confidence: 99%

Specific effect of a dopamine partial agonist on counterfactual learning: evidence from Gilles de la Tourette syndrome

Salvador

Worbe

Delorme

et al. 2017

Sci Rep

Self Cite

View full text Add to dashboard Cite

The dopamine partial agonist aripiprazole is increasingly used to treat pathologies for which other antipsychotics are indicated because it displays fewer side effects, such as sedation and depression-like symptoms, than other dopamine receptor antagonists. Previously, we showed that aripiprazole may protect motivational function by preserving reinforcement-related signals used to sustain reward-maximization. However, the effect of aripiprazole on more cognitive facets of human reinforcement learning, such as learning from the forgone outcomes of alternative courses of action (i.e., counterfactual learning), is unknown. To test the influence of aripiprazole on counterfactual learning, we administered a reinforcement learning task that involves both direct learning from obtained outcomes and indirect learning from forgone outcomes to two groups of Gilles de la Tourette (GTS) patients, one consisting of patients who were completely unmedicated and the other consisting of patients who were receiving aripiprazole monotherapy, and to healthy subjects. We found that whereas learning performance improved in the presence of counterfactual feedback in both healthy controls and unmedicated GTS patients, this was not the case in aripiprazole-medicated GTS patients. Our results suggest that whereas aripiprazole preserves direct learning of action-outcome associations, it may impair more complex inferential processes, such as counterfactual learning from forgone outcomes, in GTS patients treated with this medication.

show abstract

“…A range of neurologic (e.g., Huntington’s disease) and neuropsychiatric (e.g., obsessive–compulsive disorder, Tourette’s syndrome) disorders alter frontal BG circuitry, impacting action learning and inhibition control mechanisms (Holl, Wilkinson, Tabrizi, Painold, & Jahanshahi, 2013; Gillan et al, 2011; Worbe et al, 2011). The current learning paradigm could offer more novel insights into the effects of these conditions on action–outcome learning processes.…”

Section: Discussionmentioning

confidence: 99%

Dopamine Selectively Modulates the Outcome of Learning Unnatural Action–Valence Associations

Wouwe

Claassen

Kanoff

et al. 2017

Journal of Cognitive Neuroscience

View full text Add to dashboard Cite

Learning the contingencies between stimulus, action, and outcomes is disrupted in disorders associated with altered dopamine (DA) function in the BG, such as Parkinson disease (PD). Although the role of DA in learning to act has been extensively investigated in PD, the role of DA in “learning to withhold” (or inhibit) action to influence outcomes is not as well understood. The current study investigated the role of DA in learning to act or to withhold action to receive rewarding, or avoid punishing outcomes, in patients with PD tested “off” and “on” dopaminergic medication (n = 19) versus healthy controls (n = 30). Participants performed a reward-based learning task that orthogonalized action and outcome valence (action–reward, inaction–reward, action–punishment, inaction–punishment). We tested whether DA would bias learning toward action, toward reward, or to particular action–outcome interactions. All participants demonstrated inherent learning biases preferring action with reward and inaction to avoid punishment, and this was unaffected by medication. Instead, DA produced a complex modulation of learning less natural action–outcome associations. “Off” DA medication, patients demonstrated impairments in learning to withhold action to gain reward, suggesting a difficulty to overcome a bias toward associating inaction with punishment avoidance. On DA medication, these patterns changed, and patients showed a reduced ability to learn to act to avoid punishment, indicating a bias toward action and reward. The current findings suggest that DA in PD has a complex influence on the formation of action–outcome associations, particularly those involving less natural linkages between action and outcome valence.

show abstract

Reinforcement Learning and Gilles de la Tourette Syndrome

Cited by 49 publications

References 76 publications

An Integrative Perspective on the Role of Dopamine in Schizophrenia

An Integrative Perspective on the Role of Dopamine in Schizophrenia

Specific effect of a dopamine partial agonist on counterfactual learning: evidence from Gilles de la Tourette syndrome

Dopamine Selectively Modulates the Outcome of Learning Unnatural Action–Valence Associations

Contact Info

Product

Resources

About