Dopaminergic Drugs Modulate Learning Rates and Perseveration in Parkinson's Patients in a Dynamic Foraging Task

Abstract

Making appropriate choices often requires the ability to learn the value of available options from experience. Parkinson’s disease is characterized by a loss of dopamine neurons in the substantia nigra, neurons hypothesized to play a role in reinforcement learning. Although previous studies have shown that Parkinson’s patients are impaired in tasks involving learning from feedback, they have not directly tested the widely held hypothesis that dopamine neuron activity specifically encodes the reward prediction error signal used in reinforcement learning models. To test a key prediction of this hypothesis, we fit choice behavior from a dynamic foraging task with reinforcement learning models and show that treatment with dopaminergic drugs alters choice behavior in a manner consistent with the theory. More specifically, we found that dopaminergic drugs selectively modulate learning from positive outcomes. We observed no effect of dopaminergic drugs on learning from negative outcomes. We also found a novel dopamine-dependent effect on decision making that is not accounted for by reinforcement learning models: perseveration in choice, independent of reward history, increases with Parkinson’s disease and decreases with dopamine therapy.

Bursts of beta oscillation differentiate postperformance activity in the striatum and motor cortex of monkeys performing movement tasks

“…The exaggerated beta-band oscillations might simply crowd out the transmission of other messages. Because the durations of normal beta bursts, on the order of 50-150 ms, comport with time windows relevant to neuroplasticity and other features of neurotransmission, the bursts also could lead neural circuits to fall into fixed states, resulting in freezing or perseveration, states that are, like beta activity, decreased by dopamine replacement therapy (37). Thus, the indiscriminately high postperformance beta-band activity in Parkinson's patients could limit future choices for action by directly interfering with the normal adaptive mechanisms facilitated by postperformance beta bursts.…”

The Roles of Dopamine and Serotonin in Decision Making: Evidence from Pharmacological Experiments in Humans

“…Thus, Rutledge et al (2009) found that learning in Parkinson's disease patients on the basis of positive prediction errors was enhanced in the 'on' state compared with the 'off' state. Treatment status made no difference in learning from negative outcomes (Rutledge et al, 2009).…”

“…Thus, Rutledge et al (2009) found that learning in Parkinson's disease patients on the basis of positive prediction errors was enhanced in the 'on' state compared with the 'off' state. Treatment status made no difference in learning from negative outcomes (Rutledge et al, 2009). The most detailed investigation has been provided by Voon et al (2010), who compared the performance of three separate groups: Parkinson's patients with problem gambling and shopping behaviors, matched Parkinson's disease controls 'on' and 'off' dopamine therapy, and finally, matched normal volunteers, as a part of an fMRI protocol.…”

Impulse Control Disorders in Parkinson's Disease Are Associated with Dysfunction in Stimulus Valuation But Not Action Valuation

“…where p(c t ϭ A͉s t ) and p(c t ϭ B͉s t ) are the probability of choosing A and B, respectively, ␤ is the inverse-temperature parameter that encodes decision noise, and C t (s t , A) and C t (s t , B) represents the choice of A and B on the last presentation of s t , respectively (Lau and Glimcher, 2005;Rutledge et al, 2009). Therefore, C t (s t , A) ϭ 1 and C t (s t , B) ϭ 0 if A has been chosen in the previous presentation of s t before trial t, but if B has been chosen, C t (s t , A) ϭ 0 and C t (s t , B) ϭ 1.…”

“…It has also been shown that a popular RL model, known as Q-learning (QL), is useful for understanding the mechanistic differences in learning between on-and off-medication PD patients Rutledge et al, 2009). Although it has been hypothesized that the functional dissociation of striatal subregions is critical to understanding the underlying mechanism of compulsive behaviors in both the general population Belin et al, 2013) and PD patients (Cools et al, 2007;Dagher and Robbins, 2009), previous RL models of PD have not addressed the different roles of the ventral and dorsal striatum in the development of ICD in PD.…”