Individuals with problem gambling and obsessive-compulsive disorder learn through distinct reinforcement mechanisms

Abstract

Obsessive-compulsive disorder (OCD) and pathological gambling (PG) are accompanied by deficits in behavioural flexibility. In reinforcement learning, this inflexibility can reflect asymmetric learning from outcomes above and below expectations. In alternative frameworks, it reflects perseveration independent of learning. Here, we examine evidence for asymmetric reward-learning in OCD and PG by leveraging model-based functional magnetic resonance imaging (fMRI). Compared with healthy controls (HC), OCD patients exhibited a lower learning rate for worse-than-expected outcomes, which was associated with the attenuated encoding of negative reward prediction errors in the dorsomedial prefrontal cortex and the dorsal striatum. PG patients showed higher and lower learning rates for better- and worse-than-expected outcomes, respectively, accompanied by higher encoding of positive reward prediction errors in the anterior insula than HC. Perseveration did not differ considerably between the patient groups and HC. These findings elucidate the neural computations of reward-learning that are altered in OCD and PG, providing a potential account of behavioural inflexibility in those mental disorders.

Opponent learning with different representations in the cortico-basal ganglia pathways can develop obsession-compulsion cycle

“…Previous studies using probabilistic reversal learning tasks reported unchanged / increased learning rate for positive / negative RPEs in OCD patients [ 45 ] or increased / decreased learning rate for positive / negative RPEs in youth with OCD [ 46 ], and a recent study using a probabilistic instrumental learning task reported unchanged / decreased learning rate for positive / negative RPEs in OCD patients [ 47 ]. Reasons for these mixed results remain elusive, but our appetitive SR + aversive IR agent (large / small learning rate for positive / negative RPE in the SR-based system but the opposite pattern in the IR-based system) could potentially explain them.…”

“…Reasons for these mixed results remain elusive, but our appetitive SR + aversive IR agent (large / small learning rate for positive / negative RPE in the SR-based system but the opposite pattern in the IR-based system) could potentially explain them. Moreover, the recent study [ 47 ] has further shown that the decreased learning rate for negative RPEs in OCD was associated with attenuated representation of negative RPEs in the dorsomedial prefrontal cortex and the caudate. Given that the caudate (like dorsomedial striatum in rodents) has been implicated in model-based control (while putamen/dorsolateral striatum has been implicated in model-free control) [ 4 , 48 ], the attenuated representation of negative RPEs in the caudate in OCD can be consistent with our appetitive SR-based system having smaller learning rate for negative RPEs.…”

Learning and metacognition under volatility in GD: Lower learning rates and distorted coupling between action and confidence

“…For example, a recent study employing a probabilistic instrumental learning task with three conditions (reward, avoidance, neutral) found no overall differences in the proportion of correct choices between patients with GD and HCs in reward or avoidance trials. However, employing a computational model with two separate learning rates revealed that patients with GD exhibited relatively excessive sensitivity to positive prediction errors (PEs), but insensitivity to negative PEs ( Suzuki et al., 2023 ). These findings underscore the notion that GD might be linked to subtle and specific differences in learning rates, which might not always be easily discernible without employing sensitive experiments and computational modeling ( Hales et al., 2023 ).…”

Applying reinforcement learning to the psychopathology of obsessive-compulsive and gambling disorders: practices and pitfalls in computational model fitting

“…Furthermore, this inverse relationship between reward prediction errors and fMRI signals in the dACC was notably diminished in OCD patients. Suzuki et al (2023) explored neural computations in RL among OCD patients using a modified two-armed bandit task, where decisions to seek rewards and avoid losses were randomly interleaved across trials. Their findings demonstrated that the RL model with asymmetric learning rates more accurately predicted behavior in rewardseeking contexts for both OCD patients and healthy controls, compared to the conventional RL and RL with perseverance models.…”