Rapid decision threshold modulation by reward rate in a neural network

Simen, Patrick; Cohen, Jonathan D.; Holmes, Philip

doi:10.1016/j.neunet.2006.05.038

Cited by 108 publications

(124 citation statements)

References 53 publications

Supporting

Mentioning

121

Contrasting

Order By: Relevance

“…In the decision model, the boundary separation parameter α could be modeled as coming from control processes that respond to task demands, such as speed or accuracy instructions, as well as the accuracy of previous decisions. There are some cognitive models of these control processes, involving, for example, theories of reinforcement learning (Simen et al, 2006), or self-regulation Vickers, 1979), that could augment the decision model to generate the decision bound, and thus effectively place a prior on its possible values.…”

Section: Hierarchical Extensionmentioning

confidence: 99%

Determining informative priors for cognitive models

2017

View full text Add to dashboard Cite

The development of cognitive models involves the creative scientific formalization of assumptions, based on theory, observation, and other relevant information. In the Bayesian approach to implementing, testing, and using cognitive models, assumptions can influence both the likelihood function of the model, usually corresponding to assumptions about psychological processes, and the prior distribution over model parameters, usually corresponding to assumptions about the psychological variables that influence those processes. The specification of the prior is unique to the Bayesian context, but often raises concerns that lead to the use of vague or non-informative priors in cognitive modeling. Sometimes the concerns stem from philosophical objections, but more often practical difficulties with how priors should be determined are the stumbling block. We survey several sources of information that can help to specify priors for cognitive models, discuss some of the methods by which this information can be formalized in a prior distribution, and identify a number of benefits of including informative priors in cognitive modeling. Our discussion is based on three illustrative cognitive models, involving memory retention, categorization, and decision making.

show abstract

Section: Hierarchical Extensionmentioning

confidence: 99%

Determining informative priors for cognitive models

2017

View full text Add to dashboard Cite

show abstract

“…Importantly, the model has two specific parameters that govern two important aspects of cognitive control: one parameter reflects how carefully participants monitor performance; a second parameter describes how coarsely they adjust their behavior. 7 Another attempt to formally describe how participants adjust their behavior to meet different task constraints is made by Simen, Cohen, and Holmes (2006), who propose a neurologically plausible model of how participants' estimates of reward rates are translated to a dynamic setting of response thresholds.…”

Section: Towards a Principled Solutionmentioning

confidence: 99%

How to measure post-error slowing: A confound and a simple solution

Dutilh

Ravenzwaaij

Nieuwenhuis

et al. 2012

Journal of Mathematical Psychology

212

222

View full text Add to dashboard Cite

“…This type of model might be relevant to cases of reward-based decision making-for example, if evidence must be accumulated over time in order to decide which of several actions will yield the most reward (Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006). In these cases, reward value or probability may modulate the decision threshold (Simen, Cohen, & Holmes, 2006).…”

Section: Comparison Of Modelsmentioning

confidence: 99%

Neurocomputational mechanisms of reinforcement-guided learning in humans: A review

Cohen

2008

Cognitive, Affective, & Behavioral Neuroscience

View full text Add to dashboard Cite

Our world is filled with uncertainty and limited resources. Nearly every decision we face is clouded in uncertainty-uncertainty about the consequences of our decisions, uncertainty about whether others will obtain resources before we do, uncertainty about how different individuals will respond in similar contexts. Fortunately, we are often faced with the same or similar decision problems repeatedly, providing the opportunity to learn from our previous decisions and outcomes and to dynamically adapt decision-making strategies. The ability to use reinforcements flexibly to modify goal-directed behavior in the service of maximizing rewards and minimizing punishments is termed reinforcement learning. Theories and computational models of reinforcement learning provide a biologically and mathematically grounded framework within which to characterize, predict, and understand the b behavioral, cognitive, and neural bases of reward-guided decision making. The brain is highly adept at reinforcement learning, and although the neural processes that support this adaptation are being discovered, much remains unknown about the mechanisms by which rewards and punishments are used by the brain to optimize and guide decision making. Recent research in neuroscience and computational modeling suggests that theories of reinforcement learning provide a useful framework within which to study the behavioral and neural mechanisms of reward-based learning and decision making Camerer, 2003;Dayan & Balleine, 2002;Egelman, Person, & Montague, 1998;Montague & Berns, 2002;O'Doherty, Hampton, & Kim, 2007;Schultz, Dayan, & Montague, 1997). This literature seems to be in its infancy and yet is making noteworthy strides in chart acterizing the brain systems and neural computations that support reinforcement learning.Imaging and single-unit recording studies have demonstrated that reinforcements elicit increases in neural activity in several brain regions-notably, those in midbrain-striatal-frontal cortical circuits. Dopamine appears to be critically involved in such processes, although its precise function is still debated (Berridge, 2007;Montague, Hyman, & Cohen, 2004;Redgrave & Gurney, 2006;Schultz, 1998). Generally, changes in brain activity strongly correlate with reinforcements and their predictors, with some regions exhibiting increases in activity following rewards, as compared with losses, and some regions exhibiting increases in activity following losses, as compared with rewards. One might ask the simple question, why? Why should these brain circuits "care about" reinforcements? If the reinforcement already occurred, why continue processing and evaluatr ing its relative reward value after it has been consumed or received? The straightforward and intuitive answer is that reinforcements are useful for guiding future decisions and t actions. That is, reinforcements provide information that can be used to adjust behavior to maximize future rewards f when similar conditions or decision problems arise. Most of the neuroscience studies of rein...

show abstract

Rapid decision threshold modulation by reward rate in a neural network

Cited by 108 publications

References 53 publications

Determining informative priors for cognitive models

Determining informative priors for cognitive models

How to measure post-error slowing: A confound and a simple solution

Neurocomputational mechanisms of reinforcement-guided learning in humans: A review

Contact Info

Product

Resources

About