Recently there has been a reckoning in the dopamine field. This has suggested that the dopamine prediction error may function as a teaching signal, without endowing preceding events with value. We studied the cognitive basis of intracranial self-stimulation (ICSS), a setting where dopamine appears to be valuable. Physiological frequencies seen during reinforcement learning did not support robust ICSS or promote behavior that would indicate the stimulation was represented as a meaningful reward in a specific or general sense. This was despite demonstrating that this same physiologically-relevant signal could function as a teaching signal. However, supraphysiological frequencies supported robust ICSS where the stimulation was represented as a specific sensory event, which acted as a goal to motivate behavior. This demonstrates that dopamine neurons only support ICSS at supraphysiological frequencies, and in a manner that does not reflect our subjective experience with endogenous firing of dopamine neurons during reinforcement learning.

Behavior is often dichotomized into model-free and model-based systems1, 2. Model-free behavior prioritizes associations that have high value, regardless of the specific consequence or circumstance. In contrast, model-based behavior involves considering all possible outcomes to produce behavior that best fits the current circumstance. We typically exhibit a mixture of these behaviors so we can trade-off efficiency and flexibility. However, substance use disorder shifts behavior more strongly towards model-free systems, which produces a difficulty abstaining from drug-seeking due to an inability to withhold making the model-free high-value response3–10. The lateral hypothalamus (LH) is implicated in substance use disorder11–17and we have demonstrated that this region is critical to Pavlovian cue-reward learning18, 19. However, it is unknown whether learning occurring in LH is model-free or model-based, where the necessary teaching signal comes from to facilitate learning in LH, and whether this is relevant for learning deficits that drive substance use disorder. Here, we reveal that learning occurring in the LH is model-based. Further, we confirm the existence of an understudied projection extending from dopamine neurons in the ventral tegmental area (VTA) to the LH and demonstrate that this input underlies model-based learning in LH. Finally, we examine the impact of methamphetamine self-administration on LH-dependent model-based processes. These experiments reveal that a history of methamphetamine administration enhances the model-based control that Pavlovian cues have over decision-making, which was accompanied by a bidirectional strengthening of the LH to VTA circuit. Together, this work reveals a novel bidirectional circuit that underlies model-based learning and is relevant to the behavioral and cognitive changes that arise with substance use disorders. This circuit represents a new addition to models of addiction, which focus on instrumental components of drug addiction and increases in model-free habits after drug exposure3–10.