Retroactive modulation of spike timing-dependent plasticity by dopamine

Abstract

Most reinforcement learning models assume that the reward signal arrives after the activity that led to the reward, placing constraints on the possible underlying cellular mechanisms. Here we show that dopamine, a positive reinforcement signal, can retroactively convert hippocampal timing-dependent synaptic depression into potentiation. This effect requires functional NMDA receptors and is mediated in part through the activation of the cAMP/PKA cascade. Collectively, our results support the idea that reward-related signaling can act on a pre-established synaptic eligibility trace, thereby associating specific experiences with behaviorally distant, rewarding outcomes. This finding identifies a biologically plausible mechanism for solving the ‘distal reward problem’.DOI: http://dx.doi.org/10.7554/eLife.09685.001

A silent eligibility trace enables dopamine‐dependent synaptic plasticity for reinforcement learning in the mouse striatum

“…The consolidation of a short-term change into a more durable form on a timescale of minutes contrasts with a silent and transient eligibility trace that is not initially expressed as LTP, but may be converted to LTP if dopamine is applied in a critical time window. Similarly, in the hippocampus, Brzosko et al (2015) showed that after a prolonged period of repeated pairing, bath application of dopamine for 10-12 min during continued stimulation caused LTP. In the Brzosko et al (2015) experiments, continued stimulation was necessary and dopamine application alone, after pairing, failed to induce LTP.…”

“…In the Brzosko et al . () experiments, continued stimulation was necessary and dopamine application alone, after pairing, failed to induce LTP. The requirement for continued synaptic activity during the period of eligibility is inconsistent with the concept of a synaptic eligibility trace.…”

“…Similarly, in the hippocampus, Brzosko et al . () showed that after a prolonged period of repeated pairing, bath application of dopamine for 10–12 min during continued stimulation caused LTP. In the Brzosko et al .…”

Reverse Replay of Hippocampal Place Cells Is Uniquely Modulated by Changing Reward

“…Foster and Wilson proposed that the value assignment would be accomplished by pairing reverse replay with a reward triggered phasic dopamine signal (Foster and Wilson, 2006), with the result that downstream areas might represent a value gradient during reward approach (eg van der Meer and Redish, 2011). A second functional mechanism can be envisaged given recent reports that hippocampal dopamine can transform reverse order cell firing which normally results in LTD into LTP at short delays consistent with the interspike intervals observed in replay (Zhang et al, 2009; Brzosko et al, 2015). Indeed, the Brzosko study, performed at the CA3-CA1 synapse, suggests that under certain conditions reverse replay could actually strengthen forward associations.…”

Local Chemical Stimulation of Neurons with the Fluidic Force Microscope (FluidFM)

“…It is particularly difficult to precisely and reproducibly confine electrical stimulation because the electrodes are fairly large and the extent of the stimulated area depends on the location of axons and cell bodies on the stimulating electrodes and around them. Moreover, electrical stimulation is not able to provide neuromodulator‐specific activation of the neurons, which would be rather important for the induction of plasticity in synapses …”