Altered learning under uncertainty in unmedicated mood and anxiety disorders

Abstract

Anxiety is characterized by altered responses under uncertain conditions, but the precise mechanism by which uncertainty changes the behaviour of anxious individuals is unclear. Here we probe the computational basis of learning under uncertainty in healthy individuals and individuals with a mix of mood and anxiety disorders. Participants chose between four competing slot machines with fluctuating, reward/punishment outcomes during safety and stress. We predicted that anxious individuals under stress would learn faster about punishments, and exhibit choices that were more affected by them, formalising our predictions as parameters in reinforcement-learning accounts of behaviour. Overall, data suggest that anxious individuals are quicker to update their behaviour in response to negative outcomes (i.e. increased punishment learning-rates). When treating anxiety, it may therefore be more fruitful to encourage anxious individuals to integrate information over longer horizons when bad things happen, rather than try to blunt responses to negative outcomes.

Acute stress impairs reward learning in men

“…However, to the best of our knowledge, no studies had attempted to model how acute stress affects cognition, particularly reward and punishment learning, using such model. Another reinforcement-learning model commonly used in the stress literature includes distinct learning rates for gain and loss trials regardless of the valence of the prediction error (Aylward et al, 2019;Robinson et al, 2013;Treadway et al, 2017), meaning that learning from rewards in gain trials (or punishments in loss trials) is modeled the same way as learning from omission of rewards in gain trials (or omission of punishments in loss trials, respectively), which may not be so well supported by neurobiological evidence as a model that assumes distinct learning rates based on the prediction errors' valence (as the one we used) (Maia & Frank, 2011;O'Doherty, Dayan, Friston, Critchley, & Dolan, 2003;Schultz et al, 1997). Thus, our findings seem to contribute to a better mechanistic understanding of how acute stress may impact reward learning.…”

“…Stress is present in our day-to-day life, but, notably, how acute stress affects reward and punishment learning remains largely unknown. A growing body of evidence suggests that acute stress impairs reward-seeking behavior (Berghorst, Bogdan, Frank, & Pizzagalli, 2013;Bogdan, Perlis, Fagerness, & Pizzagalli, 2010;Bogdan, Santesso, Fagerness, Perlis, & Pizzagalli, 2011;Bogdan & Pizzagalli, 2006;Ehlers & Todd, 2017;Morris & Rottenberg, 2015;Paret & Bublatzky, 2020; but see Lighthall, Gorlick, Schoeke, Frank, & Mather, 2013), but less is known about the impact of acute stress on punishment-avoidance behavior (Aylward et al, 2019;Petzold, Plessow, Goschke, & Kirschbaum, 2010). More critically, there is even less evidence on the mechanisms that underlie the behavioral effects of acute stress on reward and punishment learning.…”

“…Computational models, such as reinforcement-learning models, are built and implemented to capture very specific cognitive and neural mechanisms, thus linking different levels of analysis, from cognitive and behavioral phenomena to neurobiological mechanisms (Chater, 2009;Daw & Frank, 2009;Frank, 2015;Nair, Rutledge, & Mason, 2020). Reinforcement-learning models are considered extremely useful tools to investigate the neural computations underpinning cognition and behavior (Collins & Frank, 2013;Daw, 2011;Daw & Frank, 2009;Huys et al, 2016;Maia & Frank, 2011;Nair et al, 2020), and can thus provide a mechanistic framework to disentangle the effects of acute stress on reward and punishment learning (Aylward et al, 2019;Huys, Pizzagalli, Bogdan, & Dayan, 2013;Luksys & Sandi, 2011;Otto, Raio, Chiang, Phelps, & Daw, 2013;Radenbach et al, 2015;Robinson, Overstreet, Charney, Vytal, & Grillon, 2013).…”

Under the Hood: Using Computational Psychiatry to Make Psychological Therapies More Mechanism-Focused

“…As such, change in computational model parameters may also serve as useful secondary outcomes for evaluating the effectiveness of psychological therapy. For example, in patients with anxiety who have measurable differences in learning rates as compared to controls (17), change in these parameters may suggest successful psychological intervention even if symptoms have not yet reached remission. This is analogous to the use of biomarkers in clinical trials being used as secondary outcomesfor example, changes in amyloid-beta measures in clinical trials for Alzheimer dementia may suggest possible treatment efficacy even if no change is seen in cognitive scores (18).…”

Anxiety makes time pass quicker: neural correlates

“…interpretation bias: Wilson et al, (2006)) and overestimate the probability and personal cost of negative events (e.g. judgement bias: Aylward et al, 2019;Charpentier et al, 2017;Mitte, 2007). While previous behavioural and neuroimaging work has mainly focused on how anxiety influences emotional information (i.e.…”