Imprecise action selection in substance use disorder: Evidence for active learning impairments when solving the explore-exploit dilemma

Abstract

Background: Substance use disorders (SUDs) are a major public health risk. However, mechanisms accounting for continued patterns of poor choices in the face of negative life consequences remain poorly understood. Methods: We use a computational (active inference) modeling approach, combined with multiple regression and hierarchical Bayesian group analyses, to examine how treatment-seeking individuals with one or more SUDs (alcohol, cannabis, sedatives, stimulants, hallucinogens, and/or opioids; N = 147) and healthy controls (HCs; N = 54) make choices to resolve uncertainty within a gambling task. A subset of SUDs (n = 49) and HCs (n = 51) propensity-matched on age, sex, and verbal IQ were also compared to replicate larger group findings. Results: Results indicate that: (a) SUDs show poorer task performance than HCs (p=.03, Cohen's d = .33), with model estimates revealing less precise action selection mechanisms (p=.004, d = .43), a lower learning rate from losses (p=.02, d = .36), and a greater learning rate from gains (p=.04, d = .31); and (b) groups do not differ significantly in goal-directed information seeking. Conclusions: Findings suggest a pattern of inconsistent behavior in response to positive outcomes in SUDs combined with a tendency to attribute negative outcomes to chance. Specifically, individuals with SUDs fail to settle on a behavior strategy despite sufficient evidence of its success. These learning impairments could help account for difficulties in adjusting behavior and maintaining optimal decision making during and after treatment.

Greater decision uncertainty characterizes a transdiagnostic patient sample during approach-avoidance conflict: a computational modelling approach

“…Because this finding occurred only in the full sample, we do not offer strong interpretations here. However, we briefly note that it may be consistent with previous studies suggesting a general blunting of brain and behavioural responses to affective stimuli in people with cocaine and methamphetamine use disorders, 74 , 75 and with studies that have linked lower self-reported sensitivity to punishment with both methamphetamine and marijuana use, 76 , 77 which could relate to continued drug use because of an insensitivity to its negative consequences (for recent computational modelling evidence supporting this possibility, see Smith and colleagues 78 ). This finding could also relate to previous work demonstrating that cognitive bias modification methods that train increased avoidance in response to alcohol cues is beneficial to recovery in an inpatient sample of people with alcoholism.…”

Lower Levels of Directed Exploration and Reflective Thinking Are Associated With Greater Anxiety and Depression

“…This finding was in the context of a volatile environment with changing reward or loss probabilities and appeared to be driven by reduced reward-seeking behavior and an elevated drive to reduce uncertainty. This is consistent with previous work showing over-exploration in depression ( 25 ), and also consistent with suggestive evidence of greater exploration with higher anxiety and depression symptoms in one of the aforementioned studies on SUDs ( 19 ). However, greater anhedonia (in patients with Schizophrenia) has also been associated with reduced exploratory behavior in prior work ( 26 ), and a more recent study reported lower directed exploration in those with higher trait somatic anxiety ( 27 ).…”

“…Reduced exploration was also observed across all participants in the context of losses relative to gains. A second study in a heterogeneous population of substance use disorder (SUD) patients found no difference in directed exploration compared to healthy participants, but found that SUDs were associated with slower learning rates from losses and greater randomness in choice [however, this was not clearly tied to random exploration; ( 19 )]. However, substance use may also influence exploration in the absence of any disorder.…”

pymdp: A Python library for active inference in discrete state spaces

“…The majority of applications have focused on cognitive neuroscience, with a particular focus on modelling decision-making under uncertainty. Nonetheless, the framework has broad applicability and has recently been applied to diverse disciplines, ranging from computational models of psychopathology [5,6,7,8], control theory [9,10,11] and reinforcement learning [12,13,14,15,16], through to social cognition [17,18,19] and even real-world engineering problems [20,21,22]. While in recent years, some of the code arising from the active inference literature has been written in open source languages like Python and Julia [23,24,25,26,16], to-date, the most popular software for simulating active inference agents is the DEM toolbox of SPM [27,28].…”

“…Policy Inference Given the definition of the expected free energy in Equations ( 6) and (7), we now are equipped to describe posterior inference over policies, i.e., how to obtain Q(π).…”