Model-Free and Model-Based Influences in Addiction-Related Behaviors

Abstract

Background: Disruptions in the decision-making processes that guide action selection are a core feature of many psychiatric disorders, including addiction. Decision-making is influenced by the goal directed and habitual systems that can be computationally characterized using model-based and model-free reinforcement learning algorithms, respectively. Recent evidence suggests an imbalance in the influence of these reinforcement-learning systems on behavior in substancedependent individuals, but it is unknown if these disruptions are a manifestation of chronic drug use and/or are a preexisting risk factor for addiction. Methods:We trained adult, male rats on a multi-stage decision-making task to quantifying model-free and model-based processes before and after self-administration of methamphetamine or saline.Results: Individual differences in model-free, but not model-based, learning prior to any drug use predicted subsequent methamphetamine self-administration: rats with lower model-free behavior took more methamphetamine than rats with higher model-free behavior. This relationship was selective to model-free updating following a rewarded, but not unrewarded, choice. Modelfree and model-based learning were both reduced in rats following methamphetamine selfadministration, which was due to a decrement in the ability of rats to use unrewarded outcomes appropriately. Moreover, the magnitude of drug-induced disruptions in model-free learning were not correlated with disruptions in model-based behavior indicating that drug self-administration independently altered both reinforcement-learning strategies.

Withdrawal from escalated cocaine self-administration impairs reversal learning by disrupting the effects of negative feedback on reward exploitation: a behavioral and computational analysis

“…Our finding that lose-shift behavior is decreased in HE rats who had some reversal experience is consistent with findings in rats exposed to methamphetamine [34,35] and in human addicts [66][67][68]. Rats self-administering methamphetamine have also been reported to show impaired learning from unrewarded outcomes, resulting in reduced model-free learning after stimulant treatment [34] and after non-contingent methamphetamine administration [35]. In addition, model-based impairments have been reported in rats during reversal [34] and habitual behavior on reinforced learning tasks has been reported in humans [67].…”

“…An important objective of this research was to investigate the nature of the widely reported impairing effects of cocaine on the flexibility of goal-directed behavior [17,28,[62][63][64][65]. Our finding that lose-shift behavior is decreased in HE rats who had some reversal experience is consistent with findings in rats exposed to methamphetamine [34,35] and in human addicts [66][67][68]. Rats self-administering methamphetamine have also been reported to show impaired learning from unrewarded outcomes, resulting in reduced model-free learning after stimulant treatment [34] and after non-contingent methamphetamine administration [35].…”

“…Rats self-administering methamphetamine have also been reported to show impaired learning from unrewarded outcomes, resulting in reduced model-free learning after stimulant treatment [34] and after non-contingent methamphetamine administration [35]. In addition, model-based impairments have been reported in rats during reversal [34] and habitual behavior on reinforced learning tasks has been reported in humans [67]. In the present study we used a model-free learning algorithm to explain performance on a spatial serial reversal task.…”

“…Using the representation of predictive relationships in the environment, acquired through trial and error, reversal learning requires animals to switch responding to a now correct stimulus while ignoring the interference of a recently rewarded, but now no-longer-correct stimulus. Reinforcement learning has been proposed as a tractable computational process underlying trial-and-error learning [30,31] with utility in modelling aspects of addiction [32,33] and stimulant administration in rodents [34,35].…”

Recruitment and disruption of ventral pallidal cue encoding during alcohol seeking

“…Addiction to alcohol and other drugs is characterized by a shift from decision‐making based on representations of outcome value (goal‐directed or “model‐based” reward‐seeking) toward stimulus‐evoked reward‐seeking based on either the incentive motivational properties of cues (Robinson & Berridge, ), the development of habits (Everitt & Robbins, ; Vandaele & Janak, ), or “model‐free” reward‐seeking (Lucantonio, Caprioli, & Schoenbaum, ; Reiter, Deserno, Wilbertz, Heinze, & Schlagenhauf, ; Sebold et al., ; but see Groman, Massi, Mathias, Lee, & Taylor, ). These shifts in decision‐making can be driven by both associative and non‐associative drug exposure.…”