A novel hypothalamic-midbrain circuit for model-based learning

Abstract

Behavior is often dichotomized into model-free and model-based systems1, 2. Model-free behavior prioritizes associations that have high value, regardless of the specific consequence or circumstance. In contrast, model-based behavior involves considering all possible outcomes to produce behavior that best fits the current circumstance. We typically exhibit a mixture of these behaviors so we can trade-off efficiency and flexibility. However, substance use disorder shifts behavior more strongly towards model-free systems, which produces a difficulty abstaining from drug-seeking due to an inability to withhold making the model-free high-value response3–10. The lateral hypothalamus (LH) is implicated in substance use disorder11–17and we have demonstrated that this region is critical to Pavlovian cue-reward learning18, 19. However, it is unknown whether learning occurring in LH is model-free or model-based, where the necessary teaching signal comes from to facilitate learning in LH, and whether this is relevant for learning deficits that drive substance use disorder. Here, we reveal that learning occurring in the LH is model-based. Further, we confirm the existence of an understudied projection extending from dopamine neurons in the ventral tegmental area (VTA) to the LH and demonstrate that this input underlies model-based learning in LH. Finally, we examine the impact of methamphetamine self-administration on LH-dependent model-based processes. These experiments reveal that a history of methamphetamine administration enhances the model-based control that Pavlovian cues have over decision-making, which was accompanied by a bidirectional strengthening of the LH to VTA circuit. Together, this work reveals a novel bidirectional circuit that underlies model-based learning and is relevant to the behavioral and cognitive changes that arise with substance use disorders. This circuit represents a new addition to models of addiction, which focus on instrumental components of drug addiction and increases in model-free habits after drug exposure3–10.

Stimulation of VTA dopamine inputs to LH upregulates orexin neuronal activity in a DRD2-dependent manner

“…The silencing of orexinergic neurons induces conditioned place preference ( Garau et al, 2020 ), suggesting that the silencing of orexin neurons is positively reinforcing. Considering that the stimulation of VTA dopamine neurons ( Harada et al, 2021 ; Pascoli et al, 2018 ) and dopaminergic terminals in the LH ( Hoang et al, 2023 ) is generally considered to be positively reinforcing, the activation of orexin neurons by dopaminergic activity might be competing with dopamine’s own positive reinforcing effect. At the moment of omission, we observed a dopamine dip both in the NAc and LH, while orexin neurons were still activated.…”

“…Blocking orexin receptors impairs Pavlovian conditioning ( Keefer et al, 2016 ), operant behavior ( Sharf et al, 2010 ), and synaptic plasticity induced by cocaine administration ( Borgland et al, 2006 ). Additionally, dopamine in the LH is essential for model-based learning, and the stimulation of dopaminergic terminals in the LH is sufficient to trigger reinforcement learning ( Hoang et al, 2023 ). These collective findings strongly suggest that the activation of orexin neurons, evoked by dopamine transients, is crucial for reinforcement learning.…”

Stimulation of VTA dopamine inputs to LH upregulates orexin neuronal activity in a DRD2-dependent manner

“…The silencing of orexinergic neurons induces conditioned place preference 78 , suggesting that the silencing of orexin neurons is positively reinforcing. Considering that the stimulation of VTA dopamine neurons 15,16 and dopaminergic terminals in the LH 79 is generally considered to be positively reinforcing, the activation of orexin neurons by dopaminergic activity might be competing with dopamine’s own positive reinforcing effect. At the moment of omission, we observed a dopamine dip both in the NAc and LH, while orexin neurons were still activated.…”

“…Blocking orexin receptors impairs pavlovian conditioning 81 , operant behavior 82 , and synaptic plasticity induced by cocaine administration 40 . Additionally, dopamine in the LH is essential for model-based learning, and the stimulation of dopaminergic terminals in the LH is sufficient to trigger reinforcement learning 79 . These collective findings strongly suggest that the activation of orexin neurons, evoked by dopamine transients, is crucial for reinforcement learning.…”