Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task

Abstract

The present investigation had two aims: (1) to study responses of dopamine neurons to stimuli with attentional and motivational significance during several steps of learning a behavioral task, and (2) to study the activity of dopamine neurons during the performance of cognitive tasks known to be impaired after lesions of these neurons. Monkeys that had previously learned a simple reaction time task were trained to perform a spatial delayed response task via two intermediate tasks. During the learning of each new task, a total of 25% of 76 dopamine neurons showed phasic responses to the delivery of primary liquid reward, whereas only 9% of 163 neurons responded to this event once task performance was established. This produced an average population response during but not after learning of each task. Reward responses during learning were significantly more numerous and pronounced in area A10, as compared to areas A8 and A9. Dopamine neurons also showed phasic responses to the two conditioned stimuli. These were the instruction cue, which was the first stimulus in each trial and indicated the target of the upcoming arm movement (58% of 76 neurons during and 44% of 163 neurons after learning), and the trigger stimulus, which was a conditioned incentive stimulus predicting reward and eliciting a saccadic eye movement and an arm reaching movement (38% of neurons during and 40% after learning). None of the dopamine neurons showed sustained activity in the delay between the instruction and trigger stimuli that would resemble the activity of neurons in dopamine terminal areas, such as the striatum and frontal cortex. Thus, dopamine neurons respond phasically to alerting external stimuli with behavioral significance whose detection is crucial for learning and performing delayed response tasks. The lack of sustained activity suggests that dopamine neurons do not encode representational processes, such as working memory, expectation of external stimuli or reward, or preparation of movement. Rather, dopamine neurons are involved with transient changes of impulse activity in basic attentional and motivational processes underlying learning and cognitive behavior.

A vector-integration-to-endpoint model for performance of viapoint movements

“…The extended arm-control model introduced below incorporates aspects of the known cortico-basal ganglionic system for eye-movement control, in particular dual excitatory and inhibitory projections from planning sites to executive sites, and execution gating realized partly by release from inhibition. This hypothetical mechanism is much different than the optimization mechanisms proposed heretofore to explain VP movements, although experimental evidence linking the basal ganglia with reinforcement learning (Schultz, Apicella & Ljungberg, 1993) suggests a way to incorporate a biology-based optimizing process.…”

The role of learning-related dopamine signals in addiction vulnerability

“…Furthermore, the longer the delay between the CS and the reward, the weaker the response (Fiorillo et al, 2008;Kobayashi and Schultz, 2008;Roesch et al, 2007), reflecting temporal discounting of future rewards. Finally, if a reward-predicting stimulus is itself preceded by another, earlier, stimulus, then the phasic activation of dopamine neurons transfers back to this earlier stimulus (Schultz et al, 1993), which is again captured by the above theoretical account (Montague et al, 1996) of model-free learning.…”

Synaptic Gating and ADHD: A Biological Theory of Comorbidity of ADHD and Anxiety

“…Sagvolden et al (2004) point out that children's behavior is gradually brought under discriminative control by the establishment of verbally governed behavior (responsive to verbal reward). Schultz et al (1993) studied the responses of dopamine neurons during the steps of learning a behavioral task. They showed that in monkeys trained to perform a spatial delayed response task, via two intermediate tasks, dopamine neurons showed responses during, but much less, after learning each task.…”