Erotic cue exposure increases physiological arousal, biases choices towards immediate rewards and attenuates model-based reinforcement learning

Abstract

Computational psychiatry focuses on identifying core cognitive processes that appear altered across a broad range of psychiatric disorders. Temporal discounting of future rewards and model-based control during reinforcement learning have proven as two promising candidates. Despite its trait-like stability, temporal discounting has been suggested to be at least partly under contextual control. For example, highly arousing cues such as erotic pictures were shown to increase discounting, although overall evidence to date remains somewhat mixed. Whether model-based reinforcement learning is similarly affected by arousing cues is unclear. Here we tested cue-reactivity effects (erotic pictures) on subsequent temporal discounting and model-based reinforcement learning in a within-subjects design in n=39 healthy male participants. Self-reported and physiological arousal (cardiac activity and pupil dilation) were assessed before and during cue exposure. Arousal was increased during exposure of erotic vs. neutral cues both on the subjective and autonomic level. Erotic cue exposure nominally increased discounting as reflected by reduced choices of delayed options. Hierarchical drift diffusion modeling (DDM) linked increased discounting to a shift in the starting point bias of evidence accumulation towards immediate options. Model-based control during reinforcement learning was reduced following erotic cues according to model-agnostic analysis. Notably, DDM linked this effect to attenuated forgetting rates of unchosen options, leaving the model-based control parameter unchanged. Our findings replicate previous work on cue-reactivity effects in temporal discounting and for the first time show similar effects in model-based reinforcement learning. Our results highlight how environmental cues can impact core human decision processes and reveal that comprehensive drift diffusion modeling approaches can yield novel insights in reward-based decision processes.

Appetitive cue exposure increases neural reward responses without modulating temporal discounting

“…On the other hand, it is consistent with a recent study from our group (Mathar et al, 2022) that used a similar cue exposure design. In Mathar et al (2022), we used psychophysiology rather than fMRI. The lack of jitter between trial phases thus allowed us to use comprehensive modeling of RT distributions using diffusion models.…”

“…Participants were instructed explicitly about the task structure and performed ten test trials during a practice run within the scanner. In accordance with previous studies (Green et al, 1997; Mathar et al, 2022; Wagner et al, 2020) we used hypothetical choice options. However, note that discount rates for real and hypothetical rewards are highly correlated and similarly processed on the neuronal level (Bickel et al, 2009; Johnson & Bickel, 2002).…”

“…Prior distributions for the group-level parameters are listed in Table 1 . Group mean priors were derived from posterior means and standard deviations from a recent study from our group, based on the Base-model (Mathar et al, 2022). STAN model code for all models is publicly available at OSF (Base-Model: osf.io/6uz8g; Offset-Model: osf.io/mgjx5).…”

“…Prior distributions for the group-level parameters are listed in Table 1. Group mean priors were derived from posterior means and standard deviations from a recent study from our group, based on the Base-model (Mathar et al, 2022).…”

“…Indeed, exposure to primary reinforcers including appetitive (erotic) cues increases activity in reward circuits, including ventral striatum (VS), orbitofrontal cortex (OFC) and ventral tegmental area (VTA; Gola et al, 2016; Markert et al, 2021; Stark et al, 2019; Wehrum-Osinsky et al, 2014). Such exposure might also lead to a bias towards short-term rewards (Li, 2008; Mathar et al, 2022; Yeomans & Brace, 2015) possibly driven by increased dopamine release. Cortical and striatal dopamine tone can modulate TD (Arrondo et al, 2015; Cools, 2008; de Wit, 2002; Hamidovic et al, 2008; Kayser et al, 2012; Petzold et al, 2019; Pine et al, 2010; Wagner et al, 2020; Weber et al, 2016).…”