Decisions bias future choices by modifying hippocampal associative memories

Abstract

Decision making is guided by memories of option values. However, retrieving items from memory renders them malleable. Here, we show that merely retrieving values from memory and making a choice between options is sufficient both to induce changes to stimulus-reward associations in the hippocampus and to bias future decision making. After allowing participants to make repeated choices between reward-conditioned stimuli, in the absence of any outcome, we observed that participants preferred stimuli they had previously chosen, and neglected previously unchosen stimuli, over otherwise identical-valued options. Using functional brain imaging, we show that decisions induced changes to hippocampal representations of stimulusoutcome associations. These changes were correlated with future decision biases. Our results indicate that choice-induced preference changes are partially driven by choice-induced modification of memory representations and suggest that merely making a choice -even without experiencing any outcomes -induces associative plasticity. Results Behavioral ExperimentsFirst, we detailed the behavioral choice-induced revaluation effect in three independent experiments. In each experiment, participants learnt associations between neutrally rated CS and three food items (Blechert, Meule, Busch, & Ohla, 2014) serving as unconditioned stimuli (US, Fig. 1A). For each participant, the US were individually chosen based on a prior rating of subjective preference on a visual analog scale ranging from low to high preference. This rating procedure resulted in selection of a low-value (US -), an intermediate-value (US 0 ) and a highvalue (US + ) food item. Next, participants rated kanji stimuli according to their subjective preference. Six of these kanjis rated in close proximity to "neutral" (center point of the visual analog scale) were selected as CS for Pavlovian learning. Two CS each were paired with one US, resulting in three categories of differently valued CS: CS + A/B, CS 0 A/B and CS -A/B, for high-, intermediate-and low-value CS.The Pavlovian learning phase was followed by a choice-induced revaluation. From two value categories, one CS each was selected, and participants made binary choices between them, interspersed with lure decisions between non-reward-predictive kanjis (Fig. 1B).Crucially, no associated US were presented following choices, excluding the possibility of alterations in strength of stimulus-outcome associations due to directly experienced outcomes.The choice-induced revaluation phase was followed by a decision probe phase in which participants chose repeatedly between all binary CS combinations to assess preferences (Fig. 1C). The key comparison was between CS presented during revaluation versus CS from the same value category that had not been presented. Again, no outcomes were presented. Decisions are biased by past choicesThere was evidence for value transfer from US to CS across all studies ( Fig. 1E-H), as indicated by significant main effects of CS value on probe phase choice probabilit...

Reinstatement of cortical outcome representations during higher-order learning

“…This reinstatement of outcome patterns occurred in a region in the lateral OFC. Notably, the exact same rostrolateral portion of OFC has previously been implicated in representing stimulus-outcome associations (Jocham et al, 2016; Klein-Flügge et al, 2013; Luettgau et al, 2020) and in correctly assigning credit for a reward to the causal stimulus choice (Jocham et al, 2016). It is also part of the network most consistently involved in taste processing as obtained from the Neurosynth data base (Yarkoni et al, 2011) and plays a well-documented role in representing motivational (Rolls, 2006, 2000; Small et al, 1999) aspects of gustatory sensation and taste memory (Kobayashi et al, 2004).…”

“…We found reinstatement of neural representations of gustatory outcomes by directly paired visual first-order CS. This reinstatement occurred in a region in the rostrolateral OFC, which has previously been implicated in representing stimulus-outcome associations (Klein-Flügge et al, 2013; Jocham et al, 2016; Luettgau et al, 2020) and in correctly assigning credit for a reward to the causal stimulus choice (Walton et al, 2010; Jocham et al, 2016). It is also part of the network most consistently involved in taste processing (Neurosynth, Yarkoni et al, 2011) and plays a well-documented role in representing motivational aspects of gustatory sensation (Small et al, 1999; Rolls, 2000, 2006) and taste memory (Kobayashi et al, 2004).…”

“…In cross-session, cross-modality classification, we employed small-volume correction ( P SVC ) to assess significance of clusters with above-chance level (0.5) classification accuracy in the lateral orbitofrontal cortex (lOFC), our a priori region-of-interest (ROI). This region has been implicated in higher-order gustatory processing, such as motivational aspects and discrimination of taste (Kobayashi et al, 2004; Miranda, 2012; Small et al, 1999), but also representation and adaptive changes of stimulus-outcome associations (Boorman et al, 2016; Jocham et al, 2016; Klein-Flügge et al, 2013; Luettgau et al, 2020). We thus reasoned that this brain region would be a well-suited candidate for associative coupling between visual conditioned stimuli and gustatory outcomes and the proposed associative transfer learning.…”

Within and Cross-Domain Effects of Choice-Induced Bias

“…(subjective preference: [3,5,6,7], perception: [8,9,10], higher cognitive inference [10,11]), and across timeframes, from immediate, trial-level effects [11,12], to long-lasting changes in preference [13]. CIB can affect both cognitive representations [14,15,16] and neural activity [17,18,19,20]. Combined, these findings suggest that the phenomenon is robust.…”

The value of confidence: Confidence prediction errors drive value-based learning in the absence of external feedback

“…Although early evidence for the theory by Brehm (17) has been challenged on methodological grounds (18,19), more recent studies have provided new support (20)(21)(22)(23)(24)(25)(26)(27). In a very recent study, Luettgau and colleagues (28) have shown that such choice-induced preference changes can also be observed for classically conditioned stimuli.…”