Prioritized experience replays on a hippocampal predictive map for learning

Abstract

Hippocampal cells are central to spatial and predictive representations, and experience replays by place cells are crucial for learning and memory. Nonetheless, how hippocampal replay patterns dynamically change during the learning process remains to be elucidated. Here, we designed a spatial task in which rats learned a new behavioral trajectory for reward. We found that as rats updated their behavioral strategies for a novel salient location, hippocampal cell ensembles increased theta-sequences and sharp wave ripple-associated synchronous spikes that preferentially replayed salient locations and reward-related contexts in reverse order. The directionality and contents of the replays progressively varied with learning, including an optimized path that had never been exploited by the animals, suggesting prioritized replays of significant experiences on a predictive map. Online feedback blockade of sharp wave ripples during a learning process inhibited stabilizing optimized behavior. These results implicate learning-associated experience replays that act to learn and reinforce specific behavioral strategies.

Experience replay is associated with efficient nonlocal learning

“…Need captures how frequent a given experience will be encountered again in the future, while gain quantifies an expected reward increase from better decisions if that experience is replayed. Consistent with this view, Igata, Ikegaya and Sasaki (28) reported that replay preferentially represents salient locations when rats update their behavioural strategies. Accordingly, we designed a novel decision-making task to measure both the behavioural effect and neural signature of nonlocal learning in humans, while at the same time manipulating need and gain to test its rational prioritisation.…”

Hippocampal CA1 Neurons Represent Positive Feedback During the Learning Process of an Associative Memory Task

“…We found that several pyramidal neurons showed choice-direction selective ( Figures 3A,B ) or reward-direction selective ( Figures 3A,C ) activity. The property of choice-direction cells might be the association between cue tone and choice ( Terada et al, 2017 ) or goal-directed encoding ( Aoki et al, 2019 ; Igata et al, 2021 ). However, the proportion of the choice-direction cells was not learning-dependent and did not significantly differ among the learning stages ( Figure 4D2 ), suggesting that their firing might reflect a stable function in hippocampal CA1 throughout the learning process, such as spatial coding of choice and/or ports.…”

Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice

“…Replay can also represent experiences that are distant in time and space (Davidson et al, 2009;Karlsson and Frank, 2009), never-used shortcuts between familiar locations (Gupta et al, 2010), and visible but inaccessible places (O ´lafsdo ´ttir et al, 2015). In addition, replay can represent an optimized route through an environment substantially before a subject begins to use the route reliably (Igata et al, 2021). These findings suggest an alternative hypothesis: that rather than informing immediately upcoming behavior, replay instead serves to maintain and link representations of certain places for later use (Dudai, 2012;Girardeau and Zugaro, 2011;Roux et al, 2017).…”

“…During sleep, replay has been linked to memory consolidation, when the repeated activation of hippocampal ensembles is thought to engage plasticity in distributed networks to facilitate long-term memory storage (Dupret et al, 2010;Ego-Stengel and Wilson, 2010;Girardeau et al, 2009;Girardeau and Zugaro, 2011;Gridchyn et al, 2020;Michon et al, 2019;van de Ven et al, 2016). The role of replay during waking is less clear, however: while awake SWRs have been linked to learning (Ferna ´ndez-Ruiz et al, 2019;Igata et al, 2021;Jadhav et al, 2012;Nokia et al, 2012), how the content of those SWRs contributes to memoryrelated information processing remains controversial.…”