Learning produces a hippocampal cognitive map in the form of an orthogonalized state machine

Abstract

Cognitive maps confer animals with flexible intelligence by representing spatial, temporal, and abstract relationships that can be used to shape thought, planning, and behavior. Cognitive maps have been observed in the hippocampus, but their algorithmic form and the processes by which they are learned remain obscure. Here, we employed large-scale, longitudinal two-photon calcium imaging to record activity from thousands of neurons in the CA1 region of the hippocampus while mice learned to efficiently collect rewards from two subtly different versions of linear tracks in virtual reality. The results provide a detailed view of the formation of a cognitive map in the hippocampus. Throughout learning, both the animal behavior and hippocampal neural activity progressed through multiple intermediate stages, gradually revealing improved task understanding and behavioral efficiency. The learning process led to progressive decorrelations in initially similar hippocampal neural activity within and across tracks, ultimately resulting in orthogonalized representations resembling a state machine capturing the inherent structure of the task. We show that a Hidden Markov Model (HMM) and a biologically plausible recurrent neural network trained using Hebbian learning can both capture core aspects of the learning dynamics and the orthogonalized representational structure in neural activity. In contrast, we show that gradient-based learning of sequence models such as Long Short-Term Memory networks (LSTMs) and Transformers do not naturally produce such representations. We further demonstrate that mice exhibited adaptive behavior in novel task settings, with neural activity reflecting flexible deployment of the state machine. These findings shed light on the mathematical form of cognitive maps, the learning rules that sculpt them, and the algorithms that promote adaptive behavior in animals. The work thus charts a course toward a deeper understanding of biological intelligence and offers insights toward developing more robust learning algorithms in artificial intelligence.

Sequential predictive learning is a unifying theory for hippocampal representation and replay

“…One of the most surprising results in our study was that spatially tuned cells are not a guarantee of a lowdimensional neural manifold with topological correspondence to the environment. Supporting this distinction between spatial tuning and a cognitive map, experimental work has shown that, while place cells appear almost instantaneously in novel environments 84 , a cognitive map only emerges after multiple days of exposure 26,28 and relies on weakly tuned cells 28,50 . In addition to predicting sequences of sensory inputs, our results suggest that a head direction signal is critical to learn a cognitive map from viewpoint-dependent observations.…”

Hippocampal sequences span experience relative to rewards

“…The significance of reward-relative sequences in the brain may lie in their connection to hidden state estimation 7,43,[161][162][163][164][165] , a concept closely related to generalization 28,166 . Hidden state estimation refers to the process by which an animal or agent infers an unobservable state, such as the probability of receiving a reward under a certain set of conditions, in order to choose an optimal behavioral strategy 161 .…”

Distinct streams for supervised and unsupervised learning in the visual cortex

“…Because leaf1 and leaf2 neural representations were similar in naive mice (Figure 2j), we hypothesized that the fine behavioral discrimination between leaf1 and leaf2 stimuli requires orthogonalization of their neural representations [27, 34]. We tested this by comparing the projections of leaf2 onto the leaf1-circle1 coding direction (Figure 3f,g).…”

“…In the mouse visual cortex, anticipatory responses to learned stimulus sequences have been interpreted as evidence for predictive coding, which is a form of unsupervised learning [22, 23]. Neural plasticity in the hippocampus has sometimes been interpreted as sensory compression [24, 25], which is also a type of unsupervised learning, though it has typically been linked to spatial rather than sensory representations [26, 27]. It is thus still not known how widely unsupervised learning may affect sensory neural representations.…”