Adolescent dopamine neurons represent reward differently during action and state guided learning

Abstract

The neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning suggesting that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar further supporting the notion that differences in reward response in each paradigm are due to differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons such as dopamine cell number and size were similar in the VTA and SN but there were age differences in baseline firing rate, stimulated release and correlated spike activity suggesting that differences in reward responsiveness by adolescent dopamine neurons are not due to differences in intrinsic properties of these neurons but engagement of different networks.

Differential Dopamine Dynamics in Adolescents and Adults

“…This suggests that any differences in dopamine neuron activity in response to reward are likely because of age-dependent differences in the network properties of these neurons. However, while there were no age-specific differences in overall learning rate, adolescents did show an increased latency to nose poke in the operant task (McCane et al, 2021, their Fig. 1B).…”

“…These results suggest that adolescents attribute higher value to rewards administered in the absence of action. Critically, these results were not because of differences in learning rates (McCane et al, 2021, their Fig. 1), or in intrinsic characteristics of the dopamine system, such as baseline firing rate, number of dopamine cells, and cell volume (McCane et al, 2021, their Fig.…”

“…Because age-dependent response patterns were incumbent on the manner in which rewards were obtained and there were no discernable differences in the intrinsic properties of adult and adolescent dopamine neurons, McCane et al (2021) next investigated whether there were changes to dopamine network dynamics that could underlie these findings. To test this, spike correlation ratios were calculated in VTA and SNc dopamine neurons, respectively, during pavlovian and operant conditioning.…”

“…Observations made by McCane et al (2021) may also help to reconcile discrepancies in how adolescents ascribe motivational value to reward-predicting cues. Some animals view the cue as a predictor and interact with the region where the reward is delivered, known as "goal tracking."…”

From Biological Synapses to “Intelligent” Robots

“…One of the ambitions of biologically inspired neural coding in models for robot control is to characterize and provide a plausible brain model for the behavioral neuroscience of reinforcement and extinction [12,13]. These are phenomena in which a behavior that has been acquired through reinforcement in operant learning decreases in strength until its full extinction when the outcome or event that reinforced it is no longer occurring (Figure 2).…”