Modeling the sub-cellular signaling pathways involved in reinforcement learning at the striatum

Wanjerkhede, Shesharao M.; Bapi, Raju S.

doi:10.1016/s0079-6123(07)68016-9

Cited by 9 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our study produces the results that match well with the experimental and simulated data such as phosphorylation of DARPP-32 (Wanjerkhede and Bapi 2009), time course of phosphorylation of CaMKII, PKA activity, and percentage of open state of NMDAR (Wanjerkhede 2006). Ahn et al (2007) have investigated the mechanism of Ca 2+ -dependent regulation of Thr-75 by PP2A, and showed that PR72 (or B or PPP2R3A) regulatory subunit of PP2A is highly expressed in striatum.…”

Section: Model Robustnesssupporting

confidence: 75%

Role of CAMKII in reinforcement learning: a computational model of glutamate and dopamine signaling pathways

Wanjerkhede

Bapi

2011

Biol Cybern

Self Cite

View full text Add to dashboard Cite

Timely release of dopamine (DA) at the striatum seems to be important for reinforcement learning (RL) mediated by the basal ganglia. Houk et al. (in: Houk et al (eds) Models of information processing in the basal ganglia, (1995) proposed a cellular signaling pathway model to characterize the interaction between DA and glutamate pathways that have a role in RL. The model simulation results, using GENESIS KINETIKIT simulator, point out that there is not only prolongation of duration as proposed by Houk et al. (1995), but also an enhancement in the amplitude of autophosphorylation of CaMKII. Further, the autophosphorylated form of CaMKII may form a basis for the "eligibility trace" condition required in RL. This simulation study is the first of its kind to support the comprehensive theoretical proposal of Houk et al. (1995).

show abstract

Section: Model Robustnesssupporting

confidence: 75%

Role of CAMKII in reinforcement learning: a computational model of glutamate and dopamine signaling pathways

Wanjerkhede

Bapi

2011

Biol Cybern

Self Cite

View full text Add to dashboard Cite

show abstract

“…A synaptic eligibility trace can used to solve this problem: synapses are tagged to receive a credit or blame signal that arrives later [8] . Synapses tagged with eligibility traces, possibly mediated by transient phosphorylations [24] or dendritic Ca 2+ currents [25] , [26] , may be reinforced by the global reinforcement signals mediated by phasic reward bursts [27] , [28] and punisher dips [29] of dopamine cell firing from ventral tegmental area (VTA) projecting to cortical areas [30] – [34] .…”

Section: Introductionmentioning

confidence: 99%

Reinforcement Learning of Targeted Movement in a Spiking Neuronal Model of Motor Cortex

et al. 2012

View full text Add to dashboard Cite

Sensorimotor control has traditionally been considered from a control theory perspective, without relation to neurobiology. In contrast, here we utilized a spiking-neuron model of motor cortex and trained it to perform a simple movement task, which consisted of rotating a single-joint “forearm” to a target. Learning was based on a reinforcement mechanism analogous to that of the dopamine system. This provided a global reward or punishment signal in response to decreasing or increasing distance from hand to target, respectively. Output was partially driven by Poisson motor babbling, creating stochastic movements that could then be shaped by learning. The virtual forearm consisted of a single segment rotated around an elbow joint, controlled by flexor and extensor muscles. The model consisted of 144 excitatory and 64 inhibitory event-based neurons, each with AMPA, NMDA, and GABA synapses. Proprioceptive cell input to this model encoded the 2 muscle lengths. Plasticity was only enabled in feedforward connections between input and output excitatory units, using spike-timing-dependent eligibility traces for synaptic credit or blame assignment. Learning resulted from a global 3-valued signal: reward (+1), no learning (0), or punishment (−1), corresponding to phasic increases, lack of change, or phasic decreases of dopaminergic cell firing, respectively. Successful learning only occurred when both reward and punishment were enabled. In this case, 5 target angles were learned successfully within 180 s of simulation time, with a median error of 8 degrees. Motor babbling allowed exploratory learning, but decreased the stability of the learned behavior, since the hand continued moving after reaching the target. Our model demonstrated that a global reinforcement signal, coupled with eligibility traces for synaptic plasticity, can train a spiking sensorimotor network to perform goal-directed motor behavior.

show abstract

“…CaMKIIα has been reported to be involved in activity-dependent dendritic plasticity in the thalamus (Willis, 2002; Wu and Cline, 1998). It is also known to be necessary for mammalian circadian rhythms generated by a hypothalamic suprachiasmatic nuclei (SCN) clock (Golombek et al, 2004) and leptin-induced plasticity in Arc nucleus (Morishita et al, 1998).…”

Section: Discussionmentioning

confidence: 99%

Distribution of CaMKIIα expression in the brain in vivo, studied by CaMKIIα-GFP mice

et al. 2013

View full text Add to dashboard Cite

To facilitate the study of the CaMKIIα function in vivo, a CaMKIIα-GFP transgenic mouse line was generated. Here, our goal is to provide the first neuroanatomical characterization of GFP expression in the CNS of this line of mouse. Overall, CaMKIIα -GFP expression is strong and highly heterogeneous, with the dentate gyrus of the hippocampus as the most abundantly expressed region. In the hippocampus, around 70% of granule and pyramidal neurons expressed strong GFP. In the neocortex, presumed pyramidal neurons were GFP positive: around 32% of layer II/III and 35% of layer VI neurons expressed GFP, and a lower expression rate was found in other layers. In the thalamus and hypothalamus, strong GFP signals were detected in the neuropil. GFP-positive cells were also found in many other regions such as the spinal trigeminal nucleus, cerebellum and basal ganglia. We further compared the GFP expression with specific antibody staining for CaMKIIα and GABA. We found that GFP+ neurons were mostly positive for CaMKIIα-IR throughout the brain, with some exceptions throughout the brain, especially in the deeper layers of neocortex. GFP and GABA-IR marked distinct neuronal populations in most brain regions with the exception of granule cells in the olfactory bulb, purkinje cells in the cerebellar, and some layer I cells in neocortex. In conclusion, GFP expression in the CaMKIIα-GFP mice is similar to the endogenous expression of CaMKIIα protein, thus these mice can be used in in vivo and in vitro physiological studies in which visualization of CaMKIIα- neuronal populations is required.

show abstract

Modeling the sub-cellular signaling pathways involved in reinforcement learning at the striatum

Cited by 9 publications

References 17 publications

Role of CAMKII in reinforcement learning: a computational model of glutamate and dopamine signaling pathways

Role of CAMKII in reinforcement learning: a computational model of glutamate and dopamine signaling pathways

Reinforcement Learning of Targeted Movement in a Spiking Neuronal Model of Motor Cortex

Distribution of CaMKIIα expression in the brain in vivo, studied by CaMKIIα-GFP mice

Contact Info

Product

Resources

About