A causal link between prediction errors, dopamine neurons and learning

Abstract

Situations where rewards are unexpectedly obtained or withheld represent opportunities for new learning. Often, this learning includes identifying cues that predict reward availability. Unexpected rewards strongly activate midbrain dopamine neurons. This phasic signal is proposed to support learning about antecedent cues by signaling discrepancies between actual and expected outcomes, termed a reward prediction error. However, it is unknown whether dopamine neuron prediction error signaling and cue-reward learning are causally linked. To test this hypothesis, we manipulated dopamine neuron activity in rats in two behavioral procedures, associative blocking and extinction, that illustrate the essential function of prediction errors in learning. We observed that optogenetic activation of dopamine neurons concurrent with reward delivery, mimicking a prediction error, was sufficient to cause long-lasting increases in cue-elicited reward-seeking behavior. Our findings establish a causal role for temporally-precise dopamine neuron signaling in cue-reward learning, bridging a critical gap between experimental evidence and influential theoretical frameworks.

The role of learning-related dopamine signals in addiction vulnerability

“…This provides behavioural evidence for the importance of prediction errors in learning. As discussed above, more direct evidence has recently been obtained by manipulating phasic dopamine signals (Steinberg et al, 2013) in an appetitive blocking design (Table 2, second row). In a variant of the blocking paradigm (transreinforcer blocking; Table 2, third row; Dickinson and Dearing 1979; Ganesan and Pearce 1988), the identity of the reinforcer is changed in the compound phase, e.g.…”

“…Abler et al 2006;Burke et al 2010;Spicer et al 2007;Tobler et al 2007) and more specific predictions of formal learning theories Kahnt et al, 2012;Rutledge et al, 2010;Tobler et al, 2007). However, it is worth keeping in mind that the haemodynamic response measured with neuroimaging is nonspecific rather than a one-to-one reflection of a particular neural event such as dopamine release (see also Düzel et al 2009), which could explain why some fMRI studies have suggested positive coding of losses (Seymour et al 2004; although see also Tom et al 2007) and a dominance of action over value (Guitart-Masip et al, 2012 (Kamin, 1969) used an aversive between-subjects design; by contrast, the experiment described in the text and depicted in abbreviated form here (Waelti et al, 2001) used an appetitive within-subject design where the test consists of a comparison between Y and X (see also Figure 2A); The optogenetic unblocking experiment of Steinberg et al (2013) used a between-subject design.…”

“…The group with active stimulation at the time of the reward showed more Pavlovian approach behaviour than the other two groups, presumably due to the additional prediction error signal elicited by optogenetically induced phasic dopamine activity. Adapted with permission from Steinberg et al (2013).…”

“…As a result, this prediction error should lead to learning about the stimulus added to the pretrained stimulus. Indeed, rats in which this kind of stimulation was active showed stronger conditioned responding to the added cue on the first test trial than rats in which active stimulation was delivered during the intertrial interval rather than the time of reward or rats in which the appropriately timed stimulation was not active (Steinberg et al, 2013). Moreover, stimulation of dopamine neurons at the usual time of reward slowed behavioural extinction (Steinberg et al, 2013).…”

“…Indeed, rats in which this kind of stimulation was active showed stronger conditioned responding to the added cue on the first test trial than rats in which active stimulation was delivered during the intertrial interval rather than the time of reward or rats in which the appropriately timed stimulation was not active (Steinberg et al, 2013). Moreover, stimulation of dopamine neurons at the usual time of reward slowed behavioural extinction (Steinberg et al, 2013). Thus the stimulation counteracted the negative prediction error induced by the absence of expected reward and thereby conditioned behaviour was sustained.…”