Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning

Phasic dopamine transmission is posited to act as a critical teaching signal that updates the stored (or "cached") values assigned to reward-predictive stimuli and actions. It is widely hypothesized that these cached values determine the selection among multiple courses of action, a premise that has provided a foundation for contemporary theories of decision making. In the current work we used fast-scan cyclic voltammetry to probe dopamine-associated cached values from cue-evoked dopamine release in the nucleus accumbens of rats performing cost-benefit decision-making paradigms to evaluate critically the relationship between dopamineassociated cached values and preferences. By manipulating the amount of effort required to obtain rewards of different sizes, we were able to bias rats toward preferring an option yielding a high-value reward in some sessions and toward instead preferring an option yielding a low-value reward in others. Therefore, this approach permitted the investigation of dopamine-associated cached values in a context in which reward magnitude and subjective preference were dissociated. We observed greater cueevoked mesolimbic dopamine release to options yielding the high-value reward even when rats preferred the option yielding the low-value reward. This result identifies a clear mismatch between the ordinal utility of the available options and the rank ordering of their cached values, thereby providing robust evidence that dopamine-associated cached values cannot be the sole determinant of choices in simple economic decision making.decision making | action selection | cached values | dopamine I n contemporary theories of economic decision making, values are assigned to reward-predictive states in which animals can take action to obtain rewards, and these state-action values are stored ("cached") for the purpose of guiding future choices based upon their rank order (1-5). It is believed that these cached values are represented as synaptic weights within corticostriatal circuitry, reflected in the activity of subpopulations of striatal projection neurons (6-9), and are updated by dopaminedependent synaptic plasticity (10-12). Indeed, a wealth of evidence suggests that the phasic activity of dopamine neurons reports instances in which current reward or expectation of future reward differs from current expectations (13-24). This pattern of activity resembles the prediction-error term from temporal-difference reinforcement-learning algorithms, which is considered the critical teaching signal for updating cached values. A notable feature of models that integrate dopamine transmission into this computational framework is that the cached value of an action is explicitly read out by the phasic dopamine response to the unexpected presentation of a cue that designates the transition into a state in which that action yields reward. Therefore, cue-evoked dopamine signaling provides a neural representation of the cached values of available actions, and if these cached values serve as the basis for action sele...

Section: Discussionmentioning

confidence: 98%

Dopamine-associated cached values are not sufficient as the basis for action selection

Hollon

Arnold²,

Gan

et al. 2014

“…Single case analysis (Table S6) indicated that these patients were most disrupted on the two distortions, but also showed that some attenuation was clearly present in the insula cases with no striatal involvement. Larger studies are needed to resolve the functional dissociations between insula and (dorsal) striatum (62). It could be reasoned that striatally mediated effects should also interfere with win processing and the personal choice manipulation (63,64), which was not the case.…”

Section: Discussionmentioning

confidence: 99%

Damage to insula abolishes cognitive distortions during simulated gambling

Clark

Studer

Bruss³

et al. 2014

109

Gambling is a naturalistic example of risky decision-making. During gambling, players typically display an array of cognitive biases that create a distorted expectancy of winning. This study investigated brain regions underpinning gambling-related cognitive distortions, contrasting patients with focal brain lesions to the ventromedial prefrontal cortex (vmPFC), insula, or amygdala ("target patients") against healthy comparison participants and lesion comparison patients (i.e., with lesions that spare the target regions). A slot machine task was used to deliver near-miss outcomes (i.e., nonwins that fall spatially close to a jackpot), and a roulette game was used to examine the gambler's fallacy (color decisions following outcome runs). Comparison groups displayed a heightened motivation to play following near misses (compared with full misses), and manifested a classic gambler's fallacy effect. Both effects were also observed in patients with vmPFC and amygdala damage, but were absent in patients with insula damage. Our findings indicate that the distorted cognitive processing of near-miss outcomes and event sequences may be ordinarily supported by the recruitment of the insula. Interventions to reduce insula reactivity could show promise in the treatment of disordered gambling.G ambling is a widespread activity with a lifetime prevalence of 78% in the United States (1) and a past-year prevalence of 73% in the United Kingdom (2). The widespread recognition that "the house always wins," reflecting the negative expected value of gambling, makes gambling an enduring puzzle for psychological and economic models of choice behavior. Cognitive approaches to gambling explain this nonnormative behavior with reference to a number of cognitive distortions and irrational beliefs that occur during gambling play, which cause the gambler to overestimate his likelihood of winning (3, 4). The illusion of control refers to how superficial features of a game, such as a choice or instrumental response, promote erroneous perceptions of skill over outcomes that are determined only by chance (5). Near-miss outcomes (nonwins that fall close to the jackpot) increase motivations to play, plausibly by fueling beliefs about skill acquisition (6). The gambler's fallacy is a bias in the processing of randomness, whereby recent consecutive outcomes are considered less likely to repeat, and conversely, outcomes that have not occurred in the recent history are perceived as "due" (7).These distortions are reliably observed in field studies, e.g., casino environments (8), and are not confined to gambling; illusory control and the gambler's fallacy are observed in stock traders (9), and near misses influence decision-making in occupational settings (10). In the laboratory, these distortions can be elicited with gambling games, allowing the comparison of these biases between different clinical groups. The overall level of distorted thinking is elevated in people with gambling problems (11,12), and these cognitions can be targeted effectively in psy...

“…Taking another perspective to interpret the Treatment × Trait interaction, we examined the relationship between trait measurement and BU Undes for OT and PL group, respectively. Under PL, there was a significant correlation between trait scores and BU Undes because individuals To examine OT effects on the dynamic learning processes of desirable and undesirable feedback, for each participant we calculated the learning rate [i.e., the strength of association between the estimation error (prediction error) and the subsequent updates, SI Appendix, SI Methods], which has been suggested as a computational principle that underlies the observed biased belief formation by pointing to estimation errors as a learning signal (45) and reflects the dynamic learning processes of prediction errors (46). The Treatment × Feedback ANOVA of collapsed data from studies 1-3 revealed a significant main effect of Feedback as participants learned to a greater degree from estimation errors in the desirable (than undesirable) trials [F(1,306) = 246.482, P < 0.001].…”

Section: Distinct Ot Effects In Individuals With High and Low Depressmentioning

confidence: 99%

Distinct oxytocin effects on belief updating in response to desirable and undesirable feedback

Wang

et al. 2016

Humans update their beliefs upon feedback and, accordingly, modify their behaviors to adapt to the complex, changing social environment. However, people tend to incorporate desirable (better than expected) feedback into their beliefs but to discount undesirable (worse than expected) feedback. Such optimistic updating has evolved as an advantageous mechanism for social adaptation. Here, we examine the role of oxytocin (OT)-an evolutionary ancient neuropeptide pivotal for social adaptation-in belief updating upon desirable and undesirable feedback in three studies (n = 320). Using a double-blind, placebo-controlled between-subjects design, we show that intranasally administered OT (IN-OT) augments optimistic belief updating by facilitating updates of desirable feedback but impairing updates of undesirable feedback. The IN-OT-induced impairment in belief updating upon undesirable feedback is more salient in individuals with high, rather than with low, depression or anxiety traits. IN-OT selectively enhances learning rate (the strength of association between estimation error and subsequent update) of desirable feedback. IN-OT also increases participants' confidence in their estimates after receiving desirable but not undesirable feedback, and the OT effect on confidence updating upon desirable feedback mediates the effect of IN-OT on optimistic belief updating. Our findings reveal distinct functional roles of OT in updating the first-order estimation and second-order confidence judgment in response to desirable and undesirable feedback, suggesting a molecular substrate for optimistic belief updating.oxytocin | social adaptation | confidence | belief updating | optimism