A Spectrum of Time Horizons for Dopamine Signals

Abstract

The striatum is critical for making decisions based on predictions of future reward. These predictions can be updated by brief pulses of dopamine, encoding reward prediction errors. However, it is unclear how this mechanism handles the need to generate predictions over multiple time horizons: from seconds or less (if singing a song) to potentially hours or more (if hunting for food). Here we monitor and model dopamine pulses across distinct striatal subregions, and find that these reflect predictions over distinct temporal scales. Dopamine dynamics systematically accelerated from ventral to dorsal-medial to dorsallateral striatum, in the tempo of their spontaneous fluctuations, their integration of prior rewards, and their discounting of future rewards. In this way parallel striatal circuits can achieve a more comprehensive set of value computations, to guide a broad range of reward-related behaviors.

Within-Mice Comparison of Microdialysis and Fiber Photometry-Recorded Dopamine Biosensor during Amphetamine Response

“…Consistent with the ability of amphetamine to block reuptake of extracellular DA through DAT, we found a strong effect on τ after amphetamine injection (Figure D). Prior to administration we found a τ of approximately 100 ms (Figure E), consistent with previously reported values from FP measurements in DS. , Within 5–10 min of administration τ rose to 300–400 ms, whereas saline-injected mice only showed a transient response to injection (Figure E). Likewise, the 99.5th percentile of the derivate, as a proxy for release rate, was affected by amphetamine (Figure F).…”

Dual credit assignment processes underlie dopamine signals in a complex spatial environment

“…This interpretation fits well with observations that lesions of NAc DA shift motivation in cost/benefit decision-making in maze tasks (45,46) and that boosting DA can immediately enhance motivation to work (17). An interesting question for future studies is how discounting future rewards over space relates to discounting over time, which has been previously reported for DA signals (47) and may involve distinct time scales in different striatal subregions (48).…”

Mesolimbic dopamine release conveys causal associations

“…Currently, we simply assume that animals set the appropriate timescale of an environment based on knowledge of the inter-reward interval. As a more principled solution, recent work has suggested that multiple parallel systems with different time constants exist in the brain and can learn a timescale invariant representation of past time ( 64 – 67 ). Third, are there as-yet unknown state space assumptions that make TDRL RPE fit our data?…”