Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis

Abstract

BackgroundDepression is characterised partly by blunted reactions to reward. However, tasks probing this deficiency have not distinguished insensitivity to reward from insensitivity to the prediction errors for reward that determine learning and are putatively reported by the phasic activity of dopamine neurons. We attempted to disentangle these factors with respect to anhedonia in the context of stress, Major Depressive Disorder (MDD), Bipolar Disorder (BPD) and a dopaminergic challenge.MethodsSix behavioural datasets involving 392 experimental sessions were subjected to a model-based, Bayesian meta-analysis. Participants across all six studies performed a probabilistic reward task that used an asymmetric reinforcement schedule to assess reward learning. Healthy controls were tested under baseline conditions, stress or after receiving the dopamine D2 agonist pramipexole. In addition, participants with current or past MDD or BPD were evaluated. Reinforcement learning models isolated the contributions of variation in reward sensitivity and learning rate.ResultsMDD and anhedonia reduced reward sensitivity more than they affected the learning rate, while a low dose of the dopamine D2 agonist pramipexole showed the opposite pattern. Stress led to a pattern consistent with a mixed effect on reward sensitivity and learning rate.ConclusionReward-related learning reflected at least two partially separable contributions. The first related to phasic prediction error signalling, and was preferentially modulated by a low dose of the dopamine agonist pramipexole. The second related directly to reward sensitivity, and was preferentially reduced in MDD and anhedonia. Stress altered both components. Collectively, these findings highlight the contribution of model-based reinforcement learning meta-analysis for dissecting anhedonic behavior.

Blunted cardiovascular reactivity during social reward anticipation in subclinical depression

“…The wealth of empirical studies strongly suggest that individuals with clinical and subclinical depression are impaired in their response to rewards both during the anticipatory (i.e., motivational) phase and during the consummatory (i.e., emotional) phase (for a detailed discussion about the distinction between these two phases, see Berridge and Robinson, 2003;Gard et al, 2006). Specifically, depressed individuals report less anticipated pleasure (e.g., Chentsova-Dutton and Hanley, 2010), show impaired reward-learning behavior (e.g., Huys et al, 2013;Liu et al, 2011;Vrieze et al, 2013), demonstrate impaired reward-related decision making (e.g., Kunisato et al, 2012;Treadway et al, 2012), have reduced activity in approach-related cortical regions (e.g., Shankman et al, 2013; for a review see Thibodeau et al, 2006), and show altered activity in reward-related brain regions (for a review see Zhang et al, 2013). Recently, we have shown reduced effort-related cardiovascular reactivity during goal pursuit in subclinical depression (Brinkmann and Franzen, 2013;Brinkmann et al, 2009;Brinkmann, 2015, 2016a).…”

The role of learning-related dopamine signals in addiction vulnerability

“…This has consequences for learning accounts of addiction as some learning tendencies appear to confer vulnerability towards developing addiction. In this part, we first present the data on individual differences in Pavlovian responding in some detail (mainly reiterating the findings of Flagel et al 2011b), then discuss its interpretation in terms of incentive salience (Berridge and Robinson, 1998;Berridge, 2004Berridge, , 2007Saunders and Robinson, 2012), and finally put forth a hypothesis that proposes a connection between the propensity to assign incentive salience and the propensity to employ model-free learning (McClure et al, 2003a;Huys et al, 2013b;Lesaint et al, 2013;Dayan and Berridge, 2013).…”

“…We now consider the hypothesis that model-free and model-based learning may at least partially map onto sign-and goal-tracking behaviour, respectively (Lesaint et al, 2013;Huys et al, 2013b). In Pavlovian conditioning experiments, reward delivery is independent of the animals' behaviour.…”

“…Several features of addiction are at least partially amenable to explanations within the overall framework outlined above. We will briefly consider partial accounts of addiction based on a) drug-induced alterations to phasic dopaminergic signals and b) individual (and druginduced) variation in the tendency to rely on model-free learning and assign incentive salience (Redish, 2004;Redish et al, 2008;Dayan, 2009;Flagel et al, 2011b;Huys et al, 2013b).…”

“…Addiction is characterised by a profound and long-lasting downregulation of dopamine D 2 receptors in the striatum (Heinz et al, 1996Volkow et al, 2009;Huys et al, 2013b), which is also characteristic of animal models of obesity (Johnson and Kenny, 2010). This downregulation may be a consequence of drug-taking, but it may also predispose to the development of addiction and to relapse (Volkow et al, 2009;Morgan et al, 2002;Thanos et al, 2001;Heinz et al, 2005;Volkow et al, 2002;Buckholtz et al, 2010).…”