Expected Reward Value and Reward Uncertainty Have Temporally Dissociable Effects on Memory Formation

Abstract

Anticipating rewards has been shown to enhance memory formation. Although substantial evidence implicates dopamine in this behavioral effect, the precise mechanisms remain ambiguous. Because dopamine nuclei have been associated with two distinct physiological signatures of reward prediction, we hypothesized two dissociable effects on memory formation. These two signatures are a phasic dopamine response immediately following a reward cue that encodes its expected value and a sustained, ramping response that has been demonstrated during high reward uncertainty [Fiorillo, C. D., Tobler, P. N., & Schultz, W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902, 2003]. Here, we show in humans that the impact of reward anticipation on memory for an event depends on its timing relative to these physiological signatures. By manipulating reward probability (100%, 50%, or 0%) and the timing of the event to be encoded (just after the reward cue versus just before expected reward outcome), we demonstrated the predicted double dissociation: Early during reward anticipation, memory formation was improved by increased expected reward value, whereas late during reward anticipation, memory formation was enhanced by reward uncertainty. Notably, although the memory benefits of high expected reward in the early interval were consolidation dependent, the memory benefits of high uncertainty in the later interval were not. These findings support the view that expected reward benefits memory consolidation via phasic dopamine release. The novel finding of a distinct memory enhancement, temporally consistent with sustained anticipatory dopamine release, points toward new mechanisms of memory modulation by reward now ripe for further investigation.

Interplay between midbrain and dorsal anterior cingulate regions arbitrates lingering reward effects on memory encoding

“…Moreover, our results support previous studies reporting better memory for information presented in association with feedback, such as positive versus negative feedback 15,16 , larger feedback prediction errors 16,21,33,34 , or the unsigned feedback prediction error 34 . However, some other studies report that memory was not modulated by feedback magnitude 1,21 or feedback prediction error 35 , or was negatively influenced by the feedback prediction error 36 . Unfortunately, these studies, although being just a handful, present a large variety in experimental parameters which may have contributed to these seemingly discrepant results.…”

“…To name a few: encoding type (incidental versus intentional), memory type (associative versus recognition memory), encodingtesting delay period (short versus long), sleep during the delay period (with versus without sleep), task relevance of memoranda, timing of stimuli, and reinforcement learning paradigm (Pavlovian versus instrumental). While the potential influence of these factors on reward-related memory have been discussed elsewhere 16,35 , we note that many of the studies reporting a positive impact of feedback value on memory formation (including ours), presented the information to be remembered (i.e., the memoranda) in close temporal vicinity to the feedback 16,21,33,34 , while those reporting no or a negative impact of feedback value presented the memoranda prior to the feedback 35,36 . For example, Jang et al 35 presented rewards either before memoranda (to induce reward anticipation), during memoranda (to elicit what the authors termed an "image prediction error"), or after memoranda (to elicit a feedback prediction error).…”

“…Thus, reward delivery might influence memory encoding in a limited time window, perhaps constrained by the rapid phasic response of dopamine neurons to reward delivery (<500 ms; 9,10 ). Yet, because a few other studies showed that activating the reward system before and after memoranda also increased subsequent memory performance 2,3 , further research needs to determine to what extent memory formation depends on the relative timing between memoranda and different aspects of reward, and how these relate to the different response profiles of the dopamine system, i.e., phasic bursts and dips, tonic activity, and sustained ramping responses 14,21 .…”

“…On the one hand, it has been proposed that average reward is encoded by tonic dopamine levels 17,18 , which may result from a sustained mode of dopamine neuron activity that is capable of up-or downregulating phasic dopamine release 19 , and would thereby modulate memory encoding. Related to this notion, one elegant study reported that the anticipation of uncertain rewards, a condition which is known to induce a sustained "ramping response" of dopamine neurons 20 , increased incidental memory encoding for images presented during the reward anticipation period 21 . On the other hand, recent evidence suggests that the VTA is under top-down control from the anterior cingulate cortex (ACC) 22,23 .…”

Reward supports flexible orienting of attention to category information and influences subsequent memory

“…Consistent with this account, reward-related memory benefits are most commonly observed in studies using delayed testing (e.g., 24 h) (Adcock, Thangavel, Whitfield-Gabrieli, Knutson, & Gabrieli, 2006;Murty, Tompary, Adcock, & Davachi, 2017;Wittmann et al, 2005). While immediate memory benefits have also been observed (Gruber, Ritchey, Wang, Doss, & Ranganath, 2016;Gruber, Watrous, Ekstrom, Ranganath, & Otten, 2013;Murty & Adcock, 2014), studies explicitly comparing across delay intervals have often observed memory benefits for delayed testing but not for immediate testing, suggesting that reward enhancement of memory may be more strongly dependent on consolidation processes (Murayama & Kitagami, 2014;Patil, Murty, Dunsmoor, Phelps, & Davachi, 2017;Stanek, Dickerson, Chiew, Clement, & Adcock, 2018;Wittmann et al, 2005).…”