The hippocampus replays experiences during quiet rest periods, and this replay benefits subsequent memory. A critical open question is how memories are prioritized for this replay. We used functional magnetic resonance imaging (fMRI) pattern analysis to track item-level replay in the hippocampus during an awake rest period after participants studied 15 objects and completed a memory test. Objects that were remembered less well were replayed more during the subsequent rest period, suggesting a prioritization process in which weaker memories—memories most vulnerable to forgetting—are selected for replay. In a second session 12 hours later, more replay of an object during a rest period predicted better subsequent memory for that object. Replay predicted memory improvement across sessions only for participants who slept during that interval. Our results provide evidence that replay in the human hippocampus prioritizes weakly learned information, predicts subsequent memory performance, and relates to memory improvement across a delay with sleep.

We employ a ‘reverse-engineering’ approach to illuminate the neurocomputational building blocks that combine to support controlled semantic cognition: the storage and context-appropriate use of conceptual knowledge. By systematically varying the structure of a computational model and assessing the functional consequences, we identified the architectural properties that best promote some core functions of the semantic system. Semantic cognition presents a challenging test case as the brain must achieve two seemingly contradictory functions: abstracting context-invariant conceptual representations across time and modalities, whilst producing specific context-sensitive behaviours appropriate for the immediate task. These functions were best achieved in models possessing a single, deep multimodal hub with sparse connections from modality-specific regions, and control systems acting on peripheral rather than deep network layers. The reverse-engineered model provides a unifying account of core findings in the cognitive neuroscience of controlled semantic cognition, including evidence from anatomy, neuropsychology, and functional brain imaging.

How does the human brain encode semantic information about objects? This paper reconciles two seemingly contradictory views. The first proposes that local neural populations independently encode semantic features; the second, that semantic representations arise as a dynamic distributed code that changes radically with stimulus processing. Combining simulations with a well-known neural network model of semantic memory, multivariate pattern classification, and human electrocorticography, we find that both views are partially correct: information about the animacy of a depicted stimulus is distributed across ventral temporal cortex in a dynamic code possessing feature-like elements posteriorly but with elements that change rapidly and nonlinearly in anterior regions. This pattern is consistent with the view that anterior temporal lobes serve as a deep cross-modal ‘hub’ in an interactive semantic network, and more generally suggests that tertiary association cortices may adopt dynamic distributed codes difficult to detect with common brain imaging methods.

We present a ‘reverse engineering’ approach to deconstruct cognition into neurocomputational mechanisms and their underlying cortical architecture, using controlled semantic cognition as a test case. By systematically varying the structure of a computational model and assessing the functional consequences, we identified architectural properties necessary for generating the core functions of the semantic system. Semantic cognition presents a challenging test case as the brain must achieve two seemingly contradictory functions: abstracting context-invariant conceptual representations across time and modalities, whilst producing specific context-sensitive behaviours appropriate for the immediate task. These functions were best achieved in models possessing a single, deep multimodal hub with sparse connections from modality-specific inputs, and control systems acting on peripheral rather than deep network layers. These architectural features correspond well with those suggested by neural data, strongly supporting the efficacy of the reverse engineering approach, and further generating novel hypotheses about the neuroanatomy of controlled semantic cognition.

No abstract