Summary Separate studies have implicated the lateral habenula (LHb) or amygdala-related regions in processing aversive stimuli, but their relationships to each other and to appetitive motivational systems are poorly understood. We show that neurons in the recently identified GABAergic rostromedial tegmental nucleus (RMTg), which receive a major LHb input, project heavily to midbrain dopamine neurons, and show phasic activations and/or Fos induction after aversive stimuli (footshocks, shock-predictive cues, food deprivation, or reward omission) and inhibitions after rewards or reward-predictive stimuli. RMTg lesions markedly reduce passive fear behaviors (freezing, open-arm avoidance) dependent on the extended amygdala, periaqueductal gray, or septum, all regions that project directly to the RMTg. In contrast, RMTg lesions spare or enhance active fear responses (treading, escape) in these same paradigms. These findings suggest that aversive inputs from widespread brain regions and stimulus modalities converge onto the RMTg, which opposes reward and motor-activating functions of midbrain dopamine neurons

Prior studies revealed that aversive stimuli and psychostimulant drugs elicit Fos expression in neurons clustered above and behind the interpeduncular nucleus that project strongly to the ventral tegmental area (VTA) and substantia nigra (SN) compacta (C). Other reports suggest that these neurons modulate responding to aversive stimuli. We now designate the region containing them as the mesopontine rostromedial tegmental nucleus (RMTg) and report herein on its neuroanatomy. Dense mu opioid receptor and somatostatin immunoreactivity characterize the RMTg, as do neurons projecting to the VTA/SNC that are enriched in GAD 67 mRNA. Strong inputs to the RMTg arise in the lateral habenula (LHb) and, to a lesser extent, the SN. Other inputs come from the frontal cortex, ventral striatopallidum, extended amygdala, septum, preoptic region, lateral, paraventricular and posterior hypothalamus, zona incerta, periaqueductal gray, intermediate layers of the contralateral superior colliculus, dorsal raphe, mesencephalic, pontine and medullary reticular formation, and the following nuclei: parafascicular, supramammillary, mammillary, ventral lateral geniculate, deep mesencephalic, red, pedunculopontine and laterodorsal tegmental, cuneiform, parabrachial and deep cerebellar. The RMTg has meager outputs to the forebrain, mainly to the ventral pallidum, preoptic-lateral hypothalamic continuum and midline-intralaminar thalamus, but much heavier outputs to the brainstem, including, most prominently, the VTA/SNC, as noted above, and to medial tegmentum, pedunculopontine and laterodorsal tegmental nuclei, dorsal raphe and the locus ceruleus and subceruleus. The RMTg may integrate multiple forebrain and brainstem inputs in relation to a dominant LHb input. Its outputs to neuromodulatory projection systems likely converge with direct LHb projections to those structures.

Lateral habenula (LHb) neurons signal negative ‘reward-prediction errors’ and inhibit midbrain dopamine (DA) neurons. Yet LHb neurons are largely glutamatergic, indicating that this inhibition may occur through an intermediate structure. Recent studies in rats have suggested a candidate for this role, the GABAergic rostromedial tegmental nucleus (RMTg), but this neural pathway has not yet been directly tested. We now show using electrophysiology and anatomic tracing that (1) the monkey has an inhibitory structure similar to the rat RMTg; (2) RMTg neurons receive excitatory input from the LHb, exhibit negative reward-prediction errors, and send axonal projections near DA soma; and (3) stimulating this structure inhibits DA neurons. Surprisingly, some RMTg neurons responded to reward cues earlier than the LHb, and carry "state value" signals not found in DA neurons. Thus, our data suggest that the RMTg translates LHb reward-prediction errors (negative) into DA reward-prediction errors (positive), while transmitting additional motivational signals to non-DA networks.

Recent evidence suggests that dopamine plays an important role in arousal, but the location of the dopaminergic neurons that may regulate arousal remains unclear. It is sometimes assumed that the dopaminergic neurons in the ventral tegmental area that project to the prefrontal cortex and striatum may regulate the state of arousal; however, the firing of these dopaminergic neurons does not correlate with overall levels of behavioral wakefulness. We identified wake-active dopaminergic neurons by combining immunohistochemical staining for Fos and tyrosine hydroxylase (TH) in awake and sleeping rats. Approximately 50% of the TH-immunoreactive (TH-ir) cells in the ventral periaqueductal gray matter (vPAG) expressed Fos protein during natural wakefulness or wakefulness induced by environmental stimulation, but none expressed Fos during sleep. Fos immunoreactivity was not seen in the substantia nigra TH-immunoreactive cells in either condition. Injections of 6-hydroxydopamine into the vPAG, which killed 55-65% of wake-active TH-ir cells but did not injure nearby serotoninergic cells, increased total daily sleep by ϳ20%. By combining retrograde and anterograde tracing, we showed that these wake-active dopaminergic cells have extensive reciprocal connections with the sleep-wake regulatory system. The vPAG dopaminergic cells may provide the long-sought ascending dopaminergic waking influence. In addition, their close relationship with the dorsal raphe nucleus will require reassessment of previous studies of the role of the dorsal raphe nucleus in sleep, because many of those experiments may have been confounded by the then-unrecognized presence of intermingled wake-active dopaminergic neurons.

Many strong rewards, including abused drugs, also produce aversive effects that are poorly understood. For example, cocaine can produce aversive conditioning after its rewarding effects have dissipated, consistent with opponent process theory, but the neural mechanisms involved are not well known. Using electrophysiological recordings in awake rats, we found that some neurons in the lateral habenula (LHb), where activation produces aversive conditioning, exhibited biphasic responses to single doses of intravenous cocaine, with an initial inhibition followed by delayed excitation paralleling cocaine's shift from rewarding to aversive. Recordings in LHb slice preparations revealed similar cocaine-induced biphasic responses and further demonstrated that biphasic responses were mimicked by dopamine, that the inhibitory phase depended on dopamine D2-like receptors, and that the delayed excitation persisted after drug washout for prolonged durations consistent with findings in vivo. c-Fos experiments further showed that cocaine-activated LHb neurons preferentially projected to and activated neurons in the rostromedial tegmental nucleus (RMTg), a recently identified target of LHb axons that is activated by negative motivational stimuli and inhibits dopamine neurons. Finally, pharmacological excitation of the RMTg produced conditioned place aversion, whereas cocaine-induced avoidance behaviors in a runway operant paradigm were abolished by lesions of LHb efferents, lesions of the RMTg, or by optogenetic inactivation of the RMTg selectively during the period when LHb neurons are activated by cocaine. Together, these results indicate that LHb/RMTg pathways contribute critically to cocaine-induced avoidance behaviors, while also participating in reciprocally inhibitory interactions with dopamine neurons.