Acetylcholine-modulated plasticity in reward-driven navigation: a computational study

Abstract

Neuromodulation plays a fundamental role in the acquisition of new behaviours. In previous experimental work, we showed that acetylcholine biases hippocampal synaptic plasticity towards depression, and the subsequent application of dopamine can retroactively convert depression into potentiation. We also demonstrated that incorporating this sequentially neuromodulated Spike-Timing-Dependent Plasticity (STDP) rule in a network model of navigation yields effective learning of changing reward locations. Here, we employ computational modelling to further characterize the effects of cholinergic depression on behaviour. We find that acetylcholine, by allowing learning from negative outcomes, enhances exploration over the action space. We show that this results in a variety of effects, depending on the structure of the model, the environment and the task. Interestingly, sequentially neuromodulated STDP also yields flexible learning, surpassing the performance of other reward-modulated plasticity rules.

Dopamine and serotonin interplay for valence-based spatial learning

“…We considered two R-STDP learning rules to model DA and 5-HT. Firstly, we implemented sequential weight change (SWC), inspired by sequentially neuromodulated plasticity (sn-Plast) (Brzosko et al, 2017;Zannone et al, 2018), in which DA produces t-LTP and 5-HT induces t-LTD during exploration. As sn-Plast, SWC is outcome-dependent and relies on the assumption that either potentiation or depression occurs after a long delay, which allows decoupling rewarding and non-rewarding trials ( Figure 1E ; Materials and methods).…”

Integration of Spiking Neural Networks for Understanding Interval Timing

“…In this same model, modulation of Ach produced the accuracy changes found from systemic injections of drugs acting on the cholinergic pathway [67]. Interestingly, Ach modulation in a hippocampal network produced variations in task precision [134] rather than accuracy. Taken together, these networks may provide evidence indicating dissociated networks for these effects.…”

“…SNNs also offer deeper insight into how different neuromodulatory pathways interact in order to produce learned behaviors. Investigating the computational roles of neuromodulated STDP in the hippocampus, researches demonstrated the importance of DA and Ach interactions on learning during a navigation task [134]. Of particular interest was the ability of Ach to enhance precision in navigation, while DA dominated learning overall.…”

Dopamine agonist treatment increases sensitivity to gamble outcomes in the hippocampus in de novo Parkinson’s disease

“…Furthermore, the hippocampus is implicated in reward-contingent memory formation in healthy subjects ( Mattfeld and Stark, 2015 ) and is densely connected with the ventral striatum ( Helmich et al, 2010 , Mattfeld and Stark, 2015 ). Taken together, the observed effects of dopamine on hippocampal activity may serve to modulate the encoding of episodes into memory ( Brzosko et al, 2015 , Brzosko et al, 2017 , Zannone et al, 2018 ).…”