Associative learning is driven by prediction errors. Dopamine transients correlate with these errors, which current interpretations limit to endowing cues with a scalar quantity reflecting the value of future rewards. Here, we tested whether dopamine might act more broadly to support learning of an associative model of the environment. Using sensory preconditioning, we show that prediction errors underlying stimulus-stimulus learning can be blocked behaviorally and reinstated by optogenetically activating dopamine neurons. We further show that suppressing the firing of these neurons across t transition prevents normal stimulus-stimulus learning. These results establish that the acquisition of model-based information about transitions between non-rewarding events is also driven by prediction errors, and that contrary to existing canon, dopamine transients are both sufficient and necessary to support this type of learning. Our findings open new possibilities for how these biological signals might support associative learning in the mammalian brain in these and other contexts.

Summary Midbrain dopamine neurons have been proposed to signal prediction errors as defined in model-free reinforcement learning algorithms. While these algorithms have been extremely powerful in interpreting dopamine activity, these models do not register any error unless there is a difference between the value of what is predicted and what is received. Yet learning often occurs in response to changes in the unique features that characterize what is received, sometimes with no change in its value at all. Here, we show that classic error-signaling dopamine neurons also respond to changes value-neutral sensory features of an expected reward. This suggests that dopamine neurons have access to a wider variety of information than contemplated by the models currently used to interpret their activity and that, while their firing may conform to predictions of these models in some cases, they are not restricted to signaling errors in the prediction of value.

SUMMARY Dopamine (DA) neurons in the ventral tegmental area (VTA) are heterogeneous and differentially regulate ingestive and locomotor behaviors that impact energy balance. Identification of which VTA DA neurons mediate behaviors that limit weight gain has been hindered, however, by the lack of molecular markers to distinguish VTA DA populations. Here, we identified a specific subset of VTA DA neurons that express neurotensin receptor-1 (NtsR1) and preferentially comprise mesolimbic, but not mesocortical, DA neurons. Genetically targeted ablation of VTA NtsR1 neurons uncouples motivated feeding and physical activity, biasing behavior toward energy expenditure and protecting mice from age-related and diet-induced weight gain. VTA NtsR1 neurons thus represent a molecularly-defined subset of DA neurons that are essential for the coordination of energy balance. Modulation of VTA NtsR1 neurons may therefore be useful to promote behaviors that prevent the development of obesity.

Dopamine neurons are proposed to signal the reward prediction error in model-free reinforcement learning algorithms. This term represents the unpredicted or 'excess' value of the rewarding event, value that is then added to the intrinsic value of any antecedent cues, contexts or events. To support this proposal, proponents cite evidence that artificiallyinduced dopamine transients cause lasting changes in behavior. Yet these studies do not generally assess learning under conditions where an endogenous prediction error would occur. Here, to address this, we conducted three experiments where we optogenetically activated dopamine neurons while rats were learning associative relationships, both with and without reward. In each experiment, the antecedent cues failed to acquire value and instead entered into associations with the later events, whether valueless cues or valued rewards. These results show that in learning situations appropriate for the appearance of a prediction error, dopamine transients support associative, rather than model-free, learning.

The basolateral amygdala (BLA) is critical for associating initially neutral cues with appetitive and aversive stimuli and receives dense neuromodulatory acetylcholine (ACh) projections. We measured BLA ACh signaling and activity of neurons expressing CaMKIIα (a marker for glutamatergic principal cells) in mice during cue-reward learning using a fluorescent ACh sensor and calcium indicators. We found that ACh levels and nucleus basalis of Meynert (NBM) cholinergic terminal activity in the BLA (NBM-BLA) increased sharply in response to reward-related events and shifted as mice learned the cue-reward contingency. BLA CaMKIIα neuron activity followed reward retrieval and moved to the reward-predictive cue after task acquisition. Optical stimulation of cholinergic NBM-BLA terminal fibers led to quicker acquisition of the cue-reward contingency. These results indicate BLA ACh signaling carries important information about salient events in cue-reward learning and provides a framework for understanding how ACh signaling contributes to shaping BLA responses to emotional stimuli.