Two-factor theory, the actor--critic model, and conditioned avoidance

Maia, Tiago V.

doi:10.3758/lb.38.1.50

Cited by 120 publications

(121 citation statements)

References 101 publications

Supporting

Mentioning

117

Contrasting

Order By: Relevance

“…Two-factor theories of active avoidance behavior (34)(35)(36) suggest that the transition from an unsafe to a safe state is coded similarly to a reward, an idea central to various notions about vigor and learning in avoidance (13). The lack of an effect of levodopa in the go to avoid losing condition is troubling for this account.…”

Section: Discussionmentioning

confidence: 99%

Action controls dopaminergic enhancement of reward representations

Guitart-Masip

Chowdhury

Sharot

et al. 2012

Proc. Natl. Acad. Sci. U.S.A.

109

144

View full text Add to dashboard Cite

Dopamine is widely observed to signal anticipation of future rewards and thus thought to be a key contributor to affectively charged decision making. However, the experiments supporting this view have not dissociated rewards from the actions that lead to, or are occasioned by, them. Here, we manipulated dopamine pharmacologically and examined the effect on a task that explicitly dissociates action and reward value. We show that dopamine enhanced the neural representation of rewarding actions, without significantly affecting the representation of reward value as such. Thus, increasing dopamine levels with levodopa selectively boosted striatal and substantia nigra/ventral tegmental representations associated with actions leading to reward, but not with actions leading to the avoidance of punishment. These findings highlight a key role for dopamine in the generation of appetitively motivated actions.functional MRI | striatum | decision-making S ubstantial evidence indicates that the neuromodulator dopamine plays at least two roles in behavioral guidance. One is to signal prediction errors regarding expected reward value (1-3), prediction errors that are also used by the striatum to guide selection of appropriate actions (4-6). A second, less emphasized, role for dopamine is to invigorate actions associated with reward (7,8). Thus, dopamine depletion results in decreased motor activity and decreased motivated behavior (9, 10) along with decreased vigor or motivation to work for rewards in demanding reinforcement schedules (7,11). These joint roles in reward prediction and motivational control can create situations that result in conflict, such as what happens when reward is attainable solely by not acting (no-go).It has been suggest that the exact opposite may be true of serotonin. That is, although the nature of any opponency between appetitive and aversive systems remains the subject of much debate (12-14), there are suggestions that serotonin acts as a mirror to dopamine and is associated with behavioral inhibition in aversive contexts (13-17). These joint roles would lead to conflict when punishment can be avoided only by acting (go).Such conflicting cases have been a focal point of recent studies where the critical manipulation involved an orthogonalization of action requirements and outcome valence (15,18). In the latter study we highlighted anticipatory representations in the striatum and substantia nigra/ventral tegmental area (SN/VTA) that reflected a dominance of action representation over an expected dominance of reward value representations, thereby hinting at a specific role for dopamine in motivation for action (18).Here, we tested the contributions of dopamine and serotonin to action and valence representations, with a specific focus on the areas we had previously highlighted, namely the striatum and SN/ VTA. First, we trained participants on a balanced 2 (reward/ punishment) × 2 (go/no-go) task that orthogonalized action and valence (18). When performance reached 95% correct choices in all conditions...

show abstract

Section: Discussionmentioning

confidence: 99%

Action controls dopaminergic enhancement of reward representations

Guitart-Masip

Chowdhury

Sharot

et al. 2012

Proc. Natl. Acad. Sci. U.S.A.

109

144

View full text Add to dashboard Cite

show abstract

“…However, few studies have addressed the question of whether dopaminergic neurotransmission in these two regions of the striatum plays a role in aversively motivated learning, such as in learning of conditioned avoidance responses (CARs). Computational models and empirical data suggest that learning actively to respond to a warning cue in order to avoid an aversive outcome depends on learning from positive prediction errors that occur when animals successfully avoid the aversive stimulus (Maia 2010;Moutoussis et al 2008). As predicted by these models, active avoidance requires dopaminergic transmission (Beninger et al 1980) and termination of an aversive stimulus is associated with phasic firing of dopamine (DA) neurons (Brischoux et al 2009).…”

Section: Introductionmentioning

confidence: 96%

Roles of D1-like dopamine receptors in the nucleus accumbens and dorsolateral striatum in conditioned avoidance responses

Wietzikoski¹,

Boschen²,

Miyoshi

et al. 2011

Psychopharmacology

View full text Add to dashboard Cite

show abstract

“…It is possible that the impulsivity-reducing effects of amphetamine reflect effects on conditioned reinforcement (Hill, 1970;Robbins, 1976) rather than effects on the appetitive component of the opportunity cost or waiting per se. Conditioned reinforcement is closely linked to the learning functions of (presumably phasic) DA, as traditionally posited in RL models such as the actor/critic (Balleine et al, 2008;Balleine and O'Doherty, 2010;Maia, 2010), and effects of amphetamine on this function might have masked the additional, performance-related effects of the opportunity cost posited by Niv et al…”

Section: Intertemporal Choicementioning

confidence: 99%

“…For instance, such events can lead to situations in which vigorous action must be taken to avoid a punishment that would otherwise occur ('active avoidance'), or, conversely, in which a prepotent action must be inhibited in order to allow a reward to occur. Effectively controlling the activation of behavior in these cases requires additional machinery for taking into account the effect of that behavior on the un-elicited punishments (or rewards) (Dayan and Huys, 2008;Boureau and Dayan, 2010;Maia, 2010). We propose that this machinery may be separate from a 5-HT system that, by itself, tightly couples aversion and inhibition because it is specialized for the more restricted set of situations, such as passive avoidance, contained in the basic model.…”

Section: The Coupling Between Inhibitory and Aversive Effects Of Seromentioning

confidence: 99%

“…RL models have traditionally envisioned that the prediction error carried by phasic DA (and, in the model, a hypothesized aversive prediction error tentatively identified with phasic 5-HT), has a role in reinforcement, by which better-than-expected outcomes increase the propensity to take the actions that led to them, and worse-than-expected outcomes decrease it (Houk et al, 1995;Balleine et al, 2008;Maia, 2010).…”

Section: Perseveration and Switchingmentioning

confidence: 99%

See 1 more Smart Citation

Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions

Cools

Nakamura²,

Daw³

2010

Neuropsychopharmacol

409

369

View full text Add to dashboard Cite

Serotonin, like dopamine (DA), has long been implicated in adaptive behavior, including decision making and reinforcement learning. However, although the two neuromodulators are tightly related and have a similar degree of functional importance, compared with DA, we have a much less specific understanding about the mechanisms by which serotonin affects behavior. Here, we draw on recent work on computational models of dopaminergic function to suggest a framework by which many of the seemingly diverse functions associated with both DA and serotoninFcomprising both affective and activational ones, as well as a number of other functions not overtly related to eitherFcan be seen as consequences of a single root mechanism.

show abstract

Two-factor theory, the actor--critic model, and conditioned avoidance

Cited by 120 publications

References 101 publications

Action controls dopaminergic enhancement of reward representations

Action controls dopaminergic enhancement of reward representations

Roles of D1-like dopamine receptors in the nucleus accumbens and dorsolateral striatum in conditioned avoidance responses

Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions

Contact Info

Product

Resources

About