Evidence that conditioned avoidance responses are reinforced by positive prediction errors signaled by tonic striatal dopamine

Dombrowski, Patrícia A.; Maia, Tiago V.; Boschen, Suelen Lúcio; Bortolanza, Mariza; Wendler, Etiéli Mara; Schwarting, Rainer K.W.; Brandão, Marcus Lira; Winn, Philip; Blaha, Charles D.; Cunha, Cláudio Da

doi:10.1016/j.bbr.2012.06.031

Cited by 36 publications

(33 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In turn, the release of fear might elicit similar neuronal mechanism as caused by positive reinforcement. Recent measurements of dopamine in the striatum during acquisition of avoidance behavior in a shuttlebox showed an increased release of dopamine during the first successful trials (Dombrowski et al, 2013). Similar release of dopamine has been observed during learning with positive reinforcers (Schultz, 2001).…”

Section: Methodsmentioning

confidence: 99%

Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning

et al. 2016

View full text Add to dashboard Cite

Biologically plausible modeling of behavioral reinforcement learning tasks has seen great improvements over the past decades. Less work has been dedicated to tasks involving contingency reversals, i.e., tasks in which the original behavioral goal is reversed one or multiple times. The ability to adjust to such reversals is a key element of behavioral flexibility. Here, we investigate the neural mechanisms underlying contingency-reversal tasks. We first conduct experiments with humans and gerbils to demonstrate memory effects, including multiple reversals in which subjects (humans and animals) show a faster learning rate when a previously learned contingency re-appears. Motivated by recurrent mechanisms of learning and memory for object categories, we propose a network architecture which involves reinforcement learning to steer an orienting system that monitors the success in reward acquisition. We suggest that a model sensory system provides feature representations which are further processed by category-related subnetworks which constitute a neural analog of expert networks. Categories are selected dynamically in a competitive field and predict the expected reward. Learning occurs in sequentialized phases to selectively focus the weight adaptation to synapses in the hierarchical network and modulate their weight changes by a global modulator signal. The orienting subsystem itself learns to bias the competition in the presence of continuous monotonic reward accumulation. In case of sudden changes in the discrepancy of predicted and acquired reward the activated motor category can be switched. We suggest that this subsystem is composed of a hierarchically organized network of dis-inhibitory mechanisms, dubbed a dynamic control network (DCN), which resembles components of the basal ganglia. The DCN selectively activates an expert network, corresponding to the current behavioral strategy. The trace of the accumulated reward is monitored such that large sudden deviations from the monotonicity of its evolution trigger a reset after which another expert subnetwork can be activated—if it has already been established before—or new categories can be recruited and associated with novel behavioral patterns.

show abstract

Section: Methodsmentioning

confidence: 99%

Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning

et al. 2016

View full text Add to dashboard Cite

show abstract

“…Studies have demonstrated impaired conditioned-avoidance learning in rats with SNc lesions induced by MPTP (Da Cunha et al, 2001; Gevaerd et al, 2001a, b; Perry et al, 2004; Bortolanza et al, 2010) or 6-OHDA (Cooper et al, 1973) and in rats with dorsal striatum lesions (Wendler et al, 2014), dorsal striatum DA depletion (Rane and King, 2011), and intra-dorsolateral striatum infusion of D1 (Wietzikoski et al, 2012) and D2 DA receptor antagonists (Boschen et al, 2011). A recent microdialysis study by Dombrowski et al (2013) found that during conditioned-avoidance learning, DA release in the striatum increased only in the first trials in which rats avoided footshocks but not after they had learned the task. However, no alteration in DA release was observed when the footshocks were presented in an unpredictable, unavoidable and inescapable manner.…”

Section: Emerging Theories In Bg Mechanismmentioning

confidence: 99%

“…In addition, the striatum and the other regions play a role in learning how to select responses instrumental to avoid aversive stimuli (Wadenberg et al, 2010; La Lumiere et al, 2005; Manago et al, 2009; Darvas et al, 2011; Dombrowski et al, 2013; Wendler et al, 2014). Striatal DA also plays a key role in associative learning (Schultz, 1997; Da Cunha et al, 2009; Niv, 2006; Bromberg-Martin et al, 2010; Da Cunha et al, 2012; Fiorillo et al, 2013; Schultz, 2013; Lak et al, 2014).…”

Section: Emerging Theories In Bg Mechanismmentioning

confidence: 99%

Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation

Cunha

Boschen

Gómez-A

et al. 2015

Neuroscience & Biobehavioral Reviews

Self Cite

View full text Add to dashboard Cite

This review presents state-of-the-art knowledge about the roles of the basal ganglia (BG) in action-selection, cognition, and motivation, and how this knowledge has been used to improve deep brain stimulation (DBS) treatment of neurological and psychiatric disorders. Such pathological conditions include Parkinson’s disease, Huntington’s disease, Tourette syndrome, depression, and obsessive-compulsive disorder. The first section presents evidence supporting current hypotheses of how the cortico-BG circuitry works to select motor and emotional actions, and how defects in this circuitry can cause symptoms of the BG diseases. Emphasis is given to the role of striatal dopamine on motor performance, motivated behaviors and learning of procedural memories. Next, the use of cutting-edge electrochemical techniques in animal and human studies of BG functioning under normal and disease conditions is discussed. Finally, functional neuroimaging studies are reviewed; these works have shown the relationship between cortico-BG structures activated during DBS and improvement of disease symptoms.

show abstract

“…In mammals, it is the key nucleus of the basal ganglia, involved in the selection of non-adverse (Darvas et al, 2011;Dombrowski et al, 2013) and pro-rewarding actions (Da Cunha et al, 2012;Kravitz et al, 2012;Yin et al, 2004). It is densely innervated with dopaminergic neurons from the midbrain nigrostriatal pathway, and its dysfunction is implicated as responsible for hypokinesia (e.g.…”

Section: The Basal Forebrainmentioning

confidence: 99%

Proliferation, Migration, and Survival of Cells in the Telencephalon of the Ball Python, Python Regius

Bales¹

View full text Add to dashboard Cite

Evidence that conditioned avoidance responses are reinforced by positive prediction errors signaled by tonic striatal dopamine

Cited by 36 publications

References 51 publications

Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning

Reversal Learning in Humans and Gerbils: Dynamic Control Network Facilitates Learning

Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation

Proliferation, Migration, and Survival of Cells in the Telencephalon of the Ball Python, Python Regius

Contact Info

Product

Resources

About