The evolving view of replay and its functions in wake and sleep

Abstract

The term hippocampal replay originally referred to the temporally compressed re-instantiation, during rest, of sequential neural activity observed during prior active wake. Since its description in the 1990’s, hippocampal replay has often been viewed as the key mechanism by which a memory trace is repeatedly rehearsed at high-speeds during sleep and gradually transferred to neocortical circuits. However, the methods used to measure the occurrence of replay remain debated, and it is now clear that the underlying neural events are considerably more complicated than the traditional narratives had suggested. “Replay-like” activity happens during wake, can play out in reverse order, may represent trajectories never taken by the animal, and may have additional functions beyond memory consolidation, from learning values and solving the problem of credit assignment to decision making and planning. Still, we know little about the role of replay in cognition, and to what extent it differs between wake and sleep. This may soon change, however, because decades-long efforts to explain replay in terms of reinforcement learning have started to yield testable predictions and possible explanations for a diverse set of observations. Here we (1) survey the diverse features of replay, focusing especially on the latest findings; (2) discuss recent attempts at unifying disparate experimental results and putatively different cognitive functions under the banner of reinforcement learning; (3) discuss methodological issues and theoretical biases that impede progress or may warrant a partial revaluation of the current literature, and finally; (4) highlight areas of considerable uncertainty and promising avenues of inquiry.

Replay in minds and machines

“…artificial agents are not bound to physical interaction with the environment and the timescales of biology. A second difference is that understanding the distinction between sleep replay and replay during wakeful pauses from active behavior has a prominence in neuroscience (and is covered extensively in previous reviews; see e.g., Findlay et al, 2021;Klinzing et al, 2019) that is not equivalently mirrored in ML research. While the contrast between sleep and wakefulness is a theme that has inspired ML research conceptually (see e.g., Hinton et al, 1995), the mere fact that artificial agents do not "sleep" in the way that biological agents do, makes it practically impossible to investigate those differences in artificial agents.…”

Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice

“…As animals move through an environment, hippocampal ''place'' cells are activated in sequence as animals pass through the ''place field'' of each cell (Eichenbaum et al, 1999;Foster and Wilson, 2006;Lee and Wilson, 2002;O'Keefe and Dostrovsky, 1971). During sleep and pauses in behavior, time-compressed versions of the same sequences of neural firing are seen, corre-sponding to retrieval of activity patterns related to the original experience (Findlay et al, 2020;Joo and Frank, 2018). Replay engages activity across many brain structures, both cortical and subcortical, suggesting that it coordinates a distributed, multimodal representation of experience (Joo and Frank, 2018).…”

A sleep schedule incorporating naps benefits the transformation of hierarchical knowledge

“…Recent accounts suggest a bidirectional link between sleep and waking cognition [ 67 ]. Similar underlying physiology such as the recently discovered awake replay-like phenomena could be involved in this process that is interesting to be explored in future work [ 68 ].…”