Reinforcement learning and dopamine in the striatum: A modeling perspective

Wanjerkhede, Shesharao M.; Bapi, Raju S.; Mytri, V. D.

doi:10.1016/j.neucom.2013.02.061

Cited by 4 publications

(6 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fig 2 shows the flowchart of this model, the parameters of the model, and available observations from the human task for each section. In this model, the decisions are made probabilistically based on In any trial (t), the value of each action (a) of the first state calculates by the weighted sum of MB (𝑄 𝑀𝐵 t ) and MF (𝑄 𝑀𝐹 𝑡 ) system value (weight: w) according to the equation (1).…”

Section: Model Structurementioning

confidence: 99%

See 1 more Smart Citation

Better Than Maximum Likelihood Estimation of Model-Based And Model-Free Learning Style

Yazdani

Vahabie

Araabi

et al. 2021

Preprint

View full text Add to dashboard Cite

Various decision-making systems work together to shape human behavior. Habitual and goal-directed systems are the two most important ones that are studied by reinforcement learning (RL), using model-free and model-based learning methods, respectively. Human behavior resembles the weighted combination of these two systems. Such a combination is modeled by the weighted sum of action values of the model-based and model-free systems. The weighting parameter has been mostly extracted by "maximum likelihood" or "maximum a-posteriori" estimation methods. In this study, we show these two well-known methods bring many challenges, and their respective extracted values are less reliable, especially in the case of limited sample size or at the proximity of extremes values. We propose that using k‑nearest neighbor, as a free format estimate, can improve the estimation error. k-nn uses global information extracted from the behavior such as stay probability, along with fitted values. The proposed method is examined by simulated experiments, where obtained results indicate the advantage of our method in reducing both bias and variance of the error. Investigation of the human behavior data from previous studies shows that the proposed method results in more statistically robust estimates in predicting other behavioral indices such as the number of gaze directions toward each target or symptoms of some psychiatric disorders. In brief, the proposed method increases the reliability of the estimated parameters and enhances the applicability of reinforcement learning paradigms in clinical trials.

show abstract

Section: Model Structurementioning

confidence: 99%

“…It is believed that multiple cognitive systems control human decision-making. Habitual and goal-directed systems are responsible for most decisions and learnings during the human's lifetime 1,2 . The habitual system constructs habits and automatic decisions, and the goaldirected system is mostly involved in the planning.…”

Section: Introductionmentioning

confidence: 99%

Better Than Maximum Likelihood Estimation of Model-Based And Model-Free Learning Style

Yazdani

Vahabie

Araabi

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Habitual and goal-directed systems are responsible for most decisions and learnings during the human's lifetime [1,2]. While the habitual system reconstructs habits and automatic decisions, the goal-directed system is involved in planning behavior of individuals.…”

Section: Introductionmentioning

confidence: 99%

Better than maximum likelihood estimation of model-based and model-free learning style

Yazdani¹,

Vahabie²,

Araabi³

et al. 2018

Preprint

View full text Add to dashboard Cite

Multiple decision making systems work together to shape the final choices in human behavior.Habitual and goal-directed systems are the two most important systems that are studied in the reinforcement learning (RL) literature by model-free and model-based learning methods. Human behavior resembles the weighted combination of these systems and such a combination is modeled by weighted summation of action`s value from the model based and model free systems. Extraction of this weighted parameter, which is important for many applications and computational modeling, has been mostly based on the maximum likelihood or maximum a posteriori methods. We show these methods bring many challenges and their respective extracted values are less reliable especially in the proximity of extremes values. We propose that using a free format learning method (k-nearest neighbor) which uses more information besides the fitted values e.g. global information like stay probability instead of trial by trial information can ameliorate the estimation error. The proposed method is examined by simulation and results show the advantage of the proposed method. In addition, investigation of the human behavior data from previous researchers proved the proposed method to result in more statistically robust results in predicting other behavioral indices such as the number of gaze directions toward each target. In brief, the proposed method increases the reliability of the estimated parameters and enhances the applicability of reinforcement learning paradigms in clinical trials.

show abstract

“…The striatum and hippocampus were chosen as these regions receive input from midbrain dopamine neurons, 30,31 and this input is thought to drive neuroplasticity and adaptive learning. 32,33 Finally, the DANA system was shown to be compact enough to monitor phasic dopamine changes, single-neuron activity, and local-field potentials in a freely moving animal. Taken together, results from these experiments demonstrate the potential of the DANA system to investigate connections between physiology, neurochemistry, and behavior.…”

mentioning

confidence: 99%

“…We characterized the instrument’s capacity to measure action potentials (600–6000 Hz) and local-field potentials (<300 Hz) in vitro and its capacity to monitor dopamine release in the striatum and single-neuron activity in the striatum or hippocampus of anesthetized rats. The striatum and hippocampus were chosen as these regions receive input from midbrain dopamine neurons, , and this input is thought to drive neuroplasticity and adaptive learning. , Finally, the DANA system was shown to be compact enough to monitor phasic dopamine changes, single-neuron activity, and local-field potentials in a freely moving animal. Taken together, results from these experiments demonstrate the potential of the DANA system to investigate connections between physiology, neurochemistry, and behavior.…”

mentioning

confidence: 99%

Platform to Enable Combined Measurement of Dopamine and Neural Activity

et al. 2017

View full text Add to dashboard Cite

Complex behaviors depend on the coordination of the activities of ensembles of neurons and the release of neuromodulators such as dopamine. The mechanisms underlying such coordination are not well-understood due to a lack of instrumentation for combined and real-time monitoring of neuromodulator release and the activities of large ensembles of neurons. Here we describe a measurement platform that allows for the combined monitoring of electrophysiology from a high-density electrode array and dopamine dynamics from a carbon-fiber microelectrode. Integration of these two measurement systems was achieved through modification of the existing instrumentation. A shared grounded reference electrode was used in both systems to minimize electrical interference. Further, an optional solid-state-relay array positioned between the electrophysiological electrode array and amplifiers was added to provide additional electrical isolation. The capacity of the integrated measurement platform, termed DANA (Dopamine And Neural Activity), to measure action potentials (high frequency) and local-field oscillations (low frequency) was characterized in vitro using an artificial cerebral spinal fluid gelatin. In vivo recordings from the DANA platform in anesthetized rats demonstrated the ability of the system for near-simultaneous measurement of dopamine release and activity from multiple neurons both in distant brain regions (striatum and hippocampus) and within the same brain region (striatum). Furthermore, this system was shown to be sufficiently compact to measure activity in freely moving animals through recording of single-neuron activity, high-frequency local-field oscillations, and dopamine release.

show abstract

Reinforcement learning and dopamine in the striatum: A modeling perspective

Cited by 4 publications

References 106 publications

Better Than Maximum Likelihood Estimation of Model-Based And Model-Free Learning Style

Better Than Maximum Likelihood Estimation of Model-Based And Model-Free Learning Style

Better than maximum likelihood estimation of model-based and model-free learning style

Platform to Enable Combined Measurement of Dopamine and Neural Activity

Contact Info

Product

Resources

About