Acetylcholine-dependent phasic dopamine activity signals exploratory locomotion and choices

Abstract

Dopamine neurons from the Ventral Tegmental Area (VTA) switch from tonic to phasic burst firing in response to reward-predictive cues and actions. Bursting is influenced by nicotinic acetylcholine receptors (nAChRs), which are not implicated in reinforcement learning, but rather in exploration and uncertainty-seeking. The leading model assigns these functions to tonic dopamine firing. To investigate this paradox, we recorded the activity of VTA dopamine neurons during a spatial decision-making task. When reward was certain, mice adopted a stereotyped behavior, and dopamine neurons signaled reward. When confronted with uncertain rewards or a novel environment, mice exhibited exploration. Modulation of phasic, but not tonic, dopamine activity predicted uncertainty-seeking and locomotor exploration. Deletion of nAChRs disrupted the influence of uncertainty and novelty on dopamine firing and behavior, sparing reward signaling and learning. Hence, nAChR modulation of dopamine neurons can influence cognitive functions on a short timescale, through the modulation of phasic, synchronous bursting.

Modeling uncertainty-seeking behavior mediated by cholinergic influence on dopamine

“…Accordingly, most computational models of this neuromodulator at the functional level focus on memory and attention related functions (Hasselmo, 2006;Pitti & Kuniyoshi, 2011;Carrere & Alexandre, 2015;Grossberg, 2017;Yu & Dayan, 2005;Avery et al, 2012). In this paper, we targeted another aspect of the cholinergic action which was highlighted in recent experimental studies (Naudé et al, 2016(Naudé et al, , 2018. These studies suggest that, through their projections to dopaminergic neurons in the ventral tegmental area, mesopontine cholinergic neurons promote exploratory uncertainty-seeking behaviors.…”

“…Alternative model 1. Acetylcholine has been reported to increase the firing rate of dopamine neurons (Naudé et al, 2016(Naudé et al, , 2018. In this model, we tested whether this feature alone could account for the difference between WT and KO animals (i.e.…”

“…• H2) dopamine modulates the decision-making network such as to implement a softmax-like rule (Daw et al, 2006;Krichmar, 2008;Cinotti et al, 2019), • H3) acetylcholine encodes the estimated uncertainty (Yu & Dayan, 2005), • H4) acetylcholine increases dopamine firing (Naudé et al, 2016(Naudé et al, , 2018, • H5) acetylcholine introduces an uncertainty bonus in the softmax-like decision rule (Naudé et al, 2016). To account for the difference between wild type (WT) mice and mice in which nicotinic acethylcholine receptors in dopamine neurons were removed (KO), as reported by Naudé et al (2016), we defined two variants of the neuromodulation component: the WT and KO variants.…”

“…The values of R ach and R da , i.e. the membrane resistance in ACh and DA neurons respectively, was accordingly set to match the mean firing rate reported by Naudé et al (2018). For ACh neurons, there was less data available and the reported frequencies are highly variable (for example, Mena-Segovia et al (2008) report firing rates from 1Hz to 30Hz in the pedunculopontine nucleus).…”

“…We modeled this decision-making process with a neural network ( Figure 1C). The hyperparameters determining the dynamics of the model were first manually set to match the mean firing rate reported by Naudé et al (2018) in DA neurons in vivo ( Figure 2B and C). The remaining hyperparameters of the model were optimized to fit the proportion of exploitative choices observed by Naudé et al (2016) in WT and KO mice ( Figure 2D).…”