A Model for Navigation in Unknown Environments Based on a Reservoir of Hippocampal Sequences

Abstract

Hippocampal place cell populations are activated in sequences on multiple time scales during active behavior, resting and sleep states, suggesting that these sequences are the genuine dynamical motifs of the hippocampal circuit. Recently, prewired hippocampal place cell sequences have even been reported to correlate to future behaviors, but so far there is no explanation of what could be the computational benefits of such a mapping between intrinsic dynamical structure and external sensory inputs. Here, I propose a computational model in which a set of predefined internal sequences is used as a dynamical reservoir to construct a spatial map of a large unknown maze based on only a small number of salient landmarks. The model is based on a new variant of temporal difference learning and implements a simultaneous localization and mapping algorithm. As a result sequences during intermittent replay periods can be decoded as spatial trajectories and improve navigation performance, which supports the functional interpretation of replay to consolidate memories of motor actions.

Directional Tuning of Phase Precession Properties in the Hippocampus

“…Phase precession is often considered a single-cell reflection of place cell sequences during theta (Dragoi and Buzsaki, 2006;Foster and Wilson, 2007;Feng et al, 2015;Leibold, 2020), and as such it should show up in peak lags of pair correlation functions, too (Dragoi and Buzsaki, 2006;Huxter et al, 2008;Geisler et al, 2010;Schlesiger et al, 2015). In 2-dimensional environments, such correlation lags have been shown to flip signs depending on the order in which a trajectory samples the place fields (Huxter et al, 2008), arguing for strong external (behavioral/sensory) drive of sequence structure.…”

Replay in minds and machines

“…Few models related to this idea have been proposed in ML, but perhaps the closest concepts are related to attractor dynamics or reservoir computing (for a review, see e.g., Lukoševičius and Jaeger, 2009). Indeed, recent computational work has suggested a link between preplay and efficient learning, arguing that attractor dynamics can account for replay (Corneil and Gerstner, 2015) or preexisting internal sequences could be used as a dynamical reservoir (Leibold, 2020). In work by Cazin et al (2019), the framework of reservoir computing is used to model the PFC that is shown to integrate replayed sequences into larger sequence assemblies that can be recalled.…”

How development sculpts hippocampal circuits and function

“…In addition, hippocampal sequences are rooted within functional circuits that are remarkably rigid against transient perturbation (260,278) and stable across days (109). It is possible that hippocampal sequences originate from a reservoir of predefined sequences wired prior to experience, as evidenced by the "preplay" phenomenon (50,77,107,159). Finally, the development of hippocampus-dependent memory is protracted and reflected by the late emergence of internally-generated sequences (91,197,216).…”