The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning

Abstract

We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that to make this possible, the hippocampal generative model is endowed with strong inductive biases to pattern-separate individual items of experience (at the first hierarchical layer), organize them into sequences (at the second layer) and then cluster them into maps (at the third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, for the sake of continual learning of multiple sequential experiences. Our experiments show that the hierarchical model using generative replay is able to learn and retain efficiently multiple spatial navigation trajectories, organizing them into separate spatial maps. Furthermore, it reproduces flexible aspects of hippocampal dynamics that have been challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in mapbased navigation, episodic memory and imagination.

Apical drive—A cellular mechanism of dreaming?

“…they are never directly experienced. An insight from recent advances in machine learning is that in order to better generalize to the environment, it is beneficial to explore these unvisited subspaces (in machine learning terms, this is known as "generative replay" (Shin et al, 2017;Stoianov et al, 2020;van de Ven et al, 2020). It is possible that dreams reflect this exploration, where a point in this internal model is activated and, via feedback connections to cortex, drives the AIZ of L5p cells ( Figure 4 , purple lines).…”

Bistability of somatic pattern memories: stochastic outcomes in bioelectric circuits underlying regeneration

“…This error-correction mechanism that eventually converges to the correct hypothesis (e.g. I am in environment 1) when prediction errors generated by one of the two competing memories are minimized [113,121,122]. In other words, reactivating spatial representations of different maps permits using them as alternative hypotheses to be tested against sensory cues and the competition settles the generative model in one hypothesis or the other.…”

The secret life of predictive brains: what's spontaneous activity for?

“…During offline periods, bottom-up stimuli are weak or absent; they cannot elicit prediction errors to correct priors, which are therefore continuously reiterated. Hence, spontaneous activity may reflect the recirculation (or resampling) of the model's priors, or the spatiotemporal patterns acquired during exposition to external stimuli [13,57,[70][71][72]. This hypothesis explains the resemblance (at the level of average statistics) between brain activations during spontaneous and evoked cortical activity [27,[32][33][34][35]37,[73][74][75][76] but it does not fully specify the content of the priors themselves.…”

“…The second kind of optimization consists in generating (sampling) fictive data from the model probability distribution, and then using these fictive data, as they were real data, to optimize the same or other models [71,112]. This method was used in an early algorithm to train unsupervised generative models: wake-sleep.…”

“…This theory inspired the widespread use of "experience replays" from a memory buffer to improve learning in deep networks [119]. The notion of "generative replay" suggests instead that hippocampal replays are generated by resampling from a generative model, not rehearsed from verbatim memories [71]. This perspective explains more naturally the fact that hippocampal replays can recombine experiences and reorganize them to follow novel learned rules [44,120] -possibly, by cross-talking with prefrontal cortex, ventral striatum and other areas [41,51].…”