Building integrated representations through interleaved learning

Abstract

Neural representations can be characterized as falling along a continuum, from distributed representations, in which neurons are responsive to many related features of the environment, to localist representations, where neurons orthogonalize activity patterns despite any input similarity. Distributed representations support powerful learning in neural network models and have been posited to exist throughout the brain, but it is unclear under what conditions humans acquire these representations and what computational advantages they may confer. In a series of behavioral experiments, we present evidence that interleaved exposure to new information facilitates the rapid formation of distributed representations in humans. As in neural network models with distributed representations, interleaved learning supports fast and automatic recognition of item relatedness, affords efficient generalization, and is especially critical for inference when learning requires statistical integration of noisy information over time. We use the data to adjudicate between several existing computational models of human memory and inference. The results demonstrate the power of interleaved learning and implicate the use of distributed representations in human inference.

A model of autonomous interactions between hippocampus and neocortex driving sleep-dependent memory consolidation

“…Our simulations demonstrate how the hippocampus can autonomously teach the neocortex new information during NREM sleep, and how alternating between NREM and REM sleep over the course of the night can promote graceful integration of new information into existing cortical knowledge. The hippocampus is implemented as our C-HORSE model, which is able to quickly learn new categories and statistics in the environment, in addition to individual episodes (29, 30, 47). We extend the model with a neocortical area that serves as the target of consolidation and with a sleep environment that allows learning and dynamics between these areas to unfold autonomously offline.…”

“…The hippocampus module in our framework is a version of our C-HORSE model (29, 30, 47), which rapidly learns both new statistical and episodic information, in its MSP and TSP, respectively. The original CLS framework proposed that the hippocampus handles the rapid synaptic changes involved in encoding new episodes, in order to prevent the interference that would occur from attempting to make these synaptic changes directly in neocortex.…”

“…Input/output layers in Simulation 1 consisted of seven feature layers, each corresponding to a satellite feature’s high level visual and verbal representation in Entorhinal Cortex (EC). Simulation 2 had a separate input and output layer, corresponding to superficial and deep EC layers (29, 30).…”

“…The model architecture for both simulations consisted of the four C-HORSE hidden layers representing the DG, CA3, pCA1, and dCA1 subfields of the hippocampus, and one neocortical hidden layer (see Supplementary Material for parameter details). Previously C-HORSE has been employed to simulate hippocampal contributions to statistical learning, associative inference, and category learning (29, 30, 47) and comes from a lineage of hippocampal models of episodic memory (16, 98). We adopted the version from Zhou et al (30), which divides CA1 into proximal and distal components.…”

Hippocampal and medial prefrontal cortices encode structural task representations following progressive and interleaved training schedules

“…Nonetheless, many solutions that do yield near-perfect performance on both premise and inferred discriminations do exist (see https://osf.io/ps3ch/ for a scripted example). With more hidden units the networks were capable of learning increasingly sparse representations of each discrimination, see [42,43]. For example, with 6 hidden units, a specific hidden unit could uniquely activate the target output for each trained discrimination.…”