Dopamine cell firing can encode errors in reward prediction, providing a learning signal to guide future behavior. Yet dopamine is also a key modulator of motivation, invigorating current behavior. Existing theories propose that fast (“phasic”) dopamine fluctuations support learning, while much slower (“tonic”) dopamine changes are involved in motivation. We examined dopamine release in the nucleus accumbens across multiple time scales, using complementary microdialysis and voltammetric methods during adaptive decision-making. We first show that minute-by-minute dopamine levels covary with reward rate and motivational vigor. We then show that second-by-second dopamine release encodes an estimate of temporally-discounted future reward (a value function). We demonstrate that changing dopamine immediately alters willingness to work, and reinforces preceding action choices by encoding temporal-difference reward prediction errors. Our results indicate that dopamine conveys a single, rapidly-evolving decision variable, the available reward for investment of effort, that is employed for both learning and motivational functions.

Summary Neuronal oscillations allow for temporal segmentation of neuronal spikes. Interdependent oscillators can integrate multiple layers of information. We examined phase-phase coupling of theta and gamma oscillators in the CA1 region of rat hippocampus during maze exploration and REM sleep. Hippocampal theta waves were asymmetric, and estimation of the spatial position of the animal was improved by identifying the waveform-based phase of spiking, compared to traditional methods used for phase estimation. Using the waveform-based theta phase, 3 distinct gamma bands were identified: slow gammaS (30-50 Hz), mid-frequency gammaM (50-90 Hz) and fast gammaF (90-150 Hz or epsilon band). The amplitude of each sub-band was modulated by the theta phase. In addition, we found reliable phase-phase coupling between theta and both gammaS and gammaM but not gammaF oscillators. We suggest that cross-frequency phase coupling can support multiple time-scale control of neuronal spikes within and across structures.

Salient cues can prompt the rapid interruption of planned actions. It has been proposed that fast, reactive behavioral inhibition involves specific basal ganglia pathways, and we tested this by comparing activity in multiple rat basal ganglia structures during performance of a stop-signal task. Subthalamic nucleus (STN) neurons showed low-latency responses to Stop cues, irrespective of whether actions were successfully canceled or not. By contrast, neurons downstream in the substantia nigra pars reticulata (SNr) responded to Stop cues only in trials with successful cancellation. Recordings and simulations together indicate that this sensorimotor gating arises from the relative timing of two distinct inputs to neurons in the SNr dorsolateral “core” subregion: cue-related excitation from STN and movement-related inhibition from striatum. Our results support race models of action cancellation, with successful stopping requiring Stop cue information to be transmitted from STN to SNr before increased striatal input creates a point of no return.

Behavioral conditioning of cue-reward pairing results in a shift of midbrain dopamine (DA) cell activity from responding to the reward to responding to the predictive cue. However, the precise time course and mechanism underlying this shift remain unclear. Here, we report a combined single-unit recording and temporal difference (TD) modeling approach to this question. The data from recordings in conscious rats showed that DA cells retain responses to predicted reward after responses to conditioned cues have developed, at least early in training. This contrasts with previous TD models that predict a gradual stepwise shift in latency with responses to rewards lost before responses develop to the conditioned cue. By exploring the TD parameter space, we demonstrate that the persistent reward responses of DA cells during conditioning are only accurately replicated by a TD model with long-lasting eligibility traces (nonzero values for the parameter ) and low learning rate (␣). These physiological constraints for TD parameters suggest that eligibility traces and low per-trial rates of plastic modification may be essential features of neural circuits for reward learning in the brain. Such properties enable rapid but stable initiation of learning when the number of stimulus-reward pairings is limited, conferring significant adaptive advantages in real-world environments.

The suppression of inappropriate actions is critical for flexible behavior. Cortical-basal ganglia networks provide key gating mechanisms for action suppression, yet the specific roles of neuronal subpopulations are poorly understood. Here, we examine Arkypallidal (“Arky”) and Prototypical (“Proto”) globus pallidus neurons during a Stop task, which requires abrupt cancellation of an imminent action. We first establish that Arky neurons can be identified by their firing properties across the natural sleep/wake cycle. We then show that Stop responses are earlier and stronger in the Arky, compared to the Proto, subpopulation. In contrast to other basal ganglia neurons, pallidal Stop responses are selective to Stop, rather than Go, cues. Furthermore, the timing of these Stop responses matches the suppression of developing striatal Go-related activity. Our results support a two-step model of action suppression: actions-in-preparation are first paused via a subthalamic-nigral pathway, then cancelled via Arky GABAergic projections to striatum.