Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals

Pike, Alexandra C.; Robinson, Oliver J.

doi:10.1001/jamapsychiatry.2022.0051

Cited by 100 publications

(84 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

Section: Discussionsupporting

confidence: 59%

“…The implications of the present findings contribute to the current literature on impairments in learning and decision-making accompanied by mood/anxiety disorders (for a review, see Pike and Robinson, 2022 ). Pike and Robinson (2022) reported that individuals with mood/anxiety disorders show higher learning rates for negative prediction errors, which may promote negative affective bias symptoms and behavioral deficits. Recently, Waselius et al (2022) demonstrated that both breathing and heartbeat rhythms influence learning and suggested that noninvasive monitoring of bodily rhythms combined with closed-loop control of external stimuli can be used to promote learning.…”

Section: Discussionsupporting

confidence: 59%

“…Therefore, a simple interpretation of the results might be that cardiac afferent signals do not affect the degree of stochasticity in the decision-making process but only the value updating process. A recent meta-analysis showed that the difference between individuals with mood/anxiety disorders and healthy control in learning was observed in learning rates, rather than inverse temperature ( Pike and Robinson, 2022 ). Considering that mood/anxiety disorders are associated with impaired interoceptive ability (for a review, see Eggart et al, 2019 ), our results suggest that the effect of interoceptive signals on learning may be specific to the value updating process.…”

Section: Discussionmentioning

confidence: 99%

See 2 more Smart Citations

Cardiac Cycle Affects the Asymmetric Value Updating in Instrumental Reward Learning

Kimura

Kanayama²,

Toyama³

et al. 2022

Front. Neurosci.

View full text Add to dashboard Cite

This study aimed to investigate whether instrumental reward learning is affected by the cardiac cycle. To this end, we examined the effects of the cardiac cycle (systole or diastole) on the computational processes underlying the participants’ choices in the instrumental learning task. In the instrumental learning task, participants were required to select one of two discriminative stimuli (neutral visual stimuli) and immediately receive reward/punishment feedback depending on the probability assigned to the chosen stimuli. To manipulate the cardiac cycle, the presentation of discriminative stimuli was timed to coincide with either cardiac systole or diastole. We fitted the participants’ choices in the task with reinforcement learning (RL) models and estimated parameters involving instrumental learning (i.e., learning rate and inverse temperature) separately in the systole and diastole trials. Model-based analysis revealed that the learning rate for positive prediction errors was higher than that for negative prediction errors in the systole trials; however, learning rates did not differ between positive and negative prediction errors in the diastole trials. These results demonstrate that the natural fluctuation of cardiac afferent signals can affect asymmetric value updating in instrumental reward learning.

show abstract

Section: Discussionsupporting

confidence: 59%

Section: Discussionsupporting

confidence: 59%

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Cardiac Cycle Affects the Asymmetric Value Updating in Instrumental Reward Learning

Kimura

Kanayama²,

Toyama³

et al. 2022

Front. Neurosci.

View full text Add to dashboard Cite

show abstract

“…It is currently an open question of whether these deficits can be explained in a common framework. For other recent reviews of the literature on anxiety and learning, we refer to Pulcu and Browning (2019) and Pike and Robinson (2022).…”

Section: Anxietymentioning

confidence: 99%

Understanding Learning Through Uncertainty and Bias

Bruckner¹,

Heekeren²,

Nassar³

2022

Preprint

View full text Add to dashboard Cite

Learning allows humans and animals to make predictions about the environment and increases the likelihood of adaptive behavior. Studying the mechanisms through which humans regulate learning to update their predictions in the face of uncertainty can also shed light on maladaptive behaviors in various clinical and age groups. Drawing on normative learning models, we illustrate how learning should be calibrated to different sources of uncertainty, including perceptual uncertainty, risk, and environmental changes. We show that such models explain many hallmarks of human learning and highlight research on their underlying neural mechanisms. Systematic deviations from normative learning, which have been explored through computational psychiatry, provide a useful lens through which to understand psychiatric illness. In particular, normative models provide important insights into why different clinical and age groups show maladaptive learning behavior, thereby attempting to pinpoint the computational cause of dysfunctional decision making. We conclude by discussing relevant goals for future research.

show abstract

“…The basic traits of the processes are extracted as free model parameters and have revealed individual differences, group differences, developmental changes, and links to specific neural substrates (Decker et al, 2016;Gillan et al, 2016;Gläscher & O'Doherty, 2010;Lee et al, 2012;Nussenbaum, 2019). For example, basic parameters such as the learning rate (which governs the updating of option values) and inverse temperature (which controls the exploration-exploitation trade-off) have often been associated with psychopathology and are expected to capture unique characteristics of some psychiatric disorders, such as depression or anxiety (Huys et al, 2021;Pike & Robinson, 2022;Robinson & Chase, 2017;Scholl & Klein-Flugge, 2018).…”

Section: Introductionmentioning

confidence: 99%

Examinations of Biases by Model Misspecification and Parameter Reliability of Reinforcement Learning Models

Toyama¹,

Katahira²,

Kunisato³

2022

Preprint

View full text Add to dashboard Cite

The influence of experience on decision-making is based mostly on outcome history but also on one’s own choice history. Reinforcement learning models can implement both processes. The influence of choice history, or perseveration, is a tendency to repeat one’s own decisions and is implemented as an explicit action autocorrelation term. However, this component is not always included in models. In the present study, we explored estimation biases caused by the lack of a perseveration term in the model on other parameters, particularly the learning rate and inverse temperature, which are critical parameters in many studies, including research in computational psychiatry. First, we examined potential estimation biases in parameters using a probabilistic learning task. This task enabled us to estimate learning rates for win and loss outcomes individually, as these outcomes have different volatility. If the model lacked a perseveration term, we observed larger estimation bias in the two learning rates and lower estimation bias in inverse temperature. The learning-rate bias slightly differed depending on the volatility of outcomes. Second, we conducted a series of simulations and found that estimation bias increased according to the magnitude of intrinsic action perseveration. Furthermore, failure to incorporate perseveration directly affected the estimation bias in learning rate and indirectly affected that in inverse temperature. In sum, this article clarifies the estimation biases in model parameters caused by failure to include perseveration and their mechanisms; we also emphasize the possible misinterpretations of results due to these biases. In addition to these model-misspecification issues, we discuss model parameter reliability, which can limit the interpretation of study results.

show abstract

Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals

Cited by 100 publications

References 77 publications

Cardiac Cycle Affects the Asymmetric Value Updating in Instrumental Reward Learning

Cardiac Cycle Affects the Asymmetric Value Updating in Instrumental Reward Learning

Understanding Learning Through Uncertainty and Bias

Examinations of Biases by Model Misspecification and Parameter Reliability of Reinforcement Learning Models

Contact Info

Product

Resources

About