Learning cognitive maps as structured graphs for vicarious evaluation

Abstract

Cognitive maps enable us to learn the layout of environments, encode and retrieve episodic memories, and navigate vicariously for mental evaluation of options. A unifying model of cognitive maps will need to 5 explain how the maps can be learned scalably with sensory observations that are non-unique over multiple spatial locations (aliased), retrieved efficiently in the face of uncertainty, and form the fabric of efficient hierarchical planning. We propose learning higher-order graphs -structured in a specific way that allows efficient learning, hierarchy formation, and inference -as the general principle that connects these different desiderata. We show that these graphs can be learned efficiently from experienced sequences using a 10 cloned Hidden Markov Model (CHMM), and uncertainty-aware planning can be achieved using messagepassing inference. Using diverse experimental settings, we show that CHMMs can be used to explain the emergence of context-specific representations, formation of transferable structural knowledge, transitive inference, shortcut finding in novel spaces, remapping of place cells, and hierarchical planning. Structured higher-order graph learning and probabilistic inference might provide a simple unifying framework for un-15 derstanding hippocampal function, and a pathway for relational abstractions in artificial intelligence.properties of place cells and grid cells [8]. Yet another recent model casts spatial and non-spatial problems as a connected graph with neural responses as efficient representations of this graph [9]. Unfortunately, 30 both these models fail to explain several experimental observations such as the discovery of place cells that encode routes [10,11], remapping in place cells [12], a recent discovery of place cells that do not encode goal value [13], and flexible planning after learning the environment.Here, we propose that learning higher-order graphs of sequential events might be an underlying principle of cognitive maps, and propose a specific representational structure that aids in learning, memory integration, 35 retrieval of episodes, and navigation. In particular, we demonstrate that this representational structure can be represented as a probabilistic sequence model -the cloned Hidden Markov Model (CHMM). We show that sequence learning in CHMMs can explain a variety of cognitive maps phenomena such as discovering spatial maps from random walks under aliased and disjoint sensory experiences, transferable structural knowledge, finding shortcuts, and hierarchical planning and physiological findings such as remapping of place cells, 40 and route-specific encoding. Notably, all these properties emerge from a simple model that is easy to train, scale, and perform inference on. Cloned Hidden Markov Model as a model of cognitive mapsCHMMs are based on Dynamic Markov Coding (DMC) [14], an idea for representing higher-order sequences by splitting, or cloning, observed states. For example, a first order Markov chain representing the 45 sequences A-C-E and B-C-D will also assi...

Learning attention-controllable border-ownership for objectness inference and binding

“…The central idea in CMRFs of using different copies of the same observation for different higher-order contexts is similar to the proposal in [36, 37]for the computational role of clonal neurons in a cortical column [38]. The CMRF can be readily instantiated as a neuronal circuit similar to the ones proposed in [36, 29]. Each clone corresponds to a neuron, and the lateral connections between these neurons make up the transition matrix.…”

“…With cloning, the same bottom-up sensory input ( C in our example) is represented by a multitude of states ( C 1 and C 2 in our example) that are copies of each other in their selectivity for the sensory input, but specialized for specific temporal contexts, enabling the efficient storage of a large number of higher-order and stochastic sequences without destructive interference. The CMRF that we present in this paper is a natural extension of the sequential analog presented in [28, 29].…”

“…All clones receive the same bottom-up or feedforward input from the observation, and they learn different representations by wiring to different lateral contexts. The output of these clones is then a weighted sum of its lateral inputs, multiplied by the bottom-up input [39, 29].…”

“…Neuroscience observations and earlier computational models guided the scaffolding of the model that allows it to learn such representations. Our model takes the form of a cloned Markov random field (CMRF) and is a natural, two-dimensional extension of the recently developed cloned hidden Markov model [28], and clone-structured cognitive graphs [29]. A key insight in this representation is clonal states [30] that represent the same bottom-up input, but learn to wire to different contexts such that higher-order contour contexts can be represented efficiently.…”

Robot navigation as hierarchical active inference

“…Research suggests rodents build topological maps, in the sense that they recall one experience being correlated by some other set of experiences through some spatial relation. This relation, typically is not only expressed in terms of distance in meters, but also in closeness through spatio-temporal consistency and experiences [5,6].…”