The formation and use of hierarchical cognitive maps in the brain: A neural network model

Abstract

Many researchers have tried to model how environmental knowledge is learned by the brain and used in the form of cognitive maps. However, previous work was limited in various important ways: there was little consensus on how these cognitive maps were formed and represented, the planning mechanism was inherently limited to performing relatively simple tasks, and there was little consideration of how these mechanisms would scale up. This paper makes several significant advances. Firstly, the planning mechanism used by the majority of previous work propagates a decaying signal through the network to create a gradient that points towards the goal. However, this decaying signal limited the scale and complexity of tasks that can be solved in this manner. Here we propose several ways in which a network can can self-organize a novel planning mechanism that does not require decaying activity. We also extend this model with a hierarchical planning mechanism: a layer of cells that identify frequently-used sequences of actions and reuse them to significantly increase the efficiency of planning. We speculate that our results may explain the apparent ability of humans and animals to perform model-based planning on both small and large scales without a noticeable loss of efficiency.

A Brain-Inspired Model of Hippocampal Spatial Cognition Based on a Memory-Replay Mechanism

“…Regarding the problem of reward signal attenuation, Mao et al [ 31 ] proposed a path planning algorithm for hierarchical reward diffusion to reduce information loss through the segmentation of environmental states. Jordan et al [ 32 ] proposed a hierarchical path representation that allows agent to perform planning of partial environmental states at a more abstract level. These models can solve the problem of signal attenuation to some extent, but they need to layer the environmental state; the operation is complex, and there is no unified standard; and there is no commonality.…”