Neural knowledge assembly in humans and deep networks

Nelli, Stephanie; Braun, Lukas; Dumbalska, Tsvetomira; Saxe, Andrew M.; Summerfield, Christopher

doi:10.1101/2021.10.21.465374

Cited by 9 publications

(15 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The resulting configuration in two dimensions (Figure 5C), chosen for intuitive visualization, exhibited a c-shaped pattern for each sequence. Similar representational geometries have previously been described in parietal cortex 63–65 . Events occurring at similar virtual times occupy similar locations, in line with high pattern similarity for events from different sequences that are separated by low temporal distances.…”

Section: Resultssupporting

confidence: 81%

Structuring time: The hippocampus constructs sequence memories that generalize temporal relations across experiences

Bellmund

Deuker

Montijn

et al. 2021

Preprint

View full text Add to dashboard Cite

The hippocampal-entorhinal region supports memory for episodic details, such as temporal relations of sequential events, and mnemonic constructions combining experiences for inferential reasoning. However, it is unclear whether hippocampal event representations reflect temporal relations derived from mnemonic constructions, event order, or elapsing time, and whether they generalize temporal relations across similar sequences. Here, participants mnemonically constructed times of events from multiple sequences using infrequent cues and their experience of passing time. After learning, event representations in the anterior hippocampus reflected sequence relations based on constructed times. These event representations generalized across sequences, revealing distinct representational formats for events from the same or different sequences. Structural knowledge about time patterns, abstracted from different sequences, biased the construction of specific event times. These findings demonstrate that the hippocampus reconciles representations of specific relations with the generalization across different episodes, consistent with memory-based constructions combining episodic details and general knowledge to simulate scenarios.

show abstract

Section: Resultssupporting

confidence: 81%

Structuring time: The hippocampus constructs sequence memories that generalize temporal relations across experiences

Bellmund

Deuker

Montijn

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…By contrast, we observed that neural manifolds representing space were highly aligned across contexts in most brain regions. This resembles the "neural structure alignment" that has recently been reported to accompany decision tasks in both humans and monkeys, whereby contexts sharing common structure are represented with parallel neural geometries, potentially because this allows a decoder trained in one context to be generalised to the other [46][47][48][49][50][51].…”

Section: Discussionmentioning

confidence: 91%

Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex

Muhle-Karbe

Sheahan

Pezzulo

et al. 2023

Preprint

View full text Add to dashboard Cite

Humans can navigate flexibly to meet their goals. Here, we asked how the neural representation of allocentric space is distorted by goal-directed behaviour. Participants navigated an agent to two successive goal locations in a grid world environment comprising four interlinked rooms, with a contextual cue indicating the conditional dependence of one goal location on another. Examining the neural geometry by which room and context were encoded in fMRI signals, we found that map-like representations of the environment emerged in both hippocampus and neocortex. Cognitive maps in hippocampus and orbitofrontal cortices were compressed so that locations cued as goals were coded together in neural state space, and these distortions predicted successful learning. This effect was captured by a computational model in which current and prospective locations are jointly encoded in a place code, providing a theory of how goals warp the neural representation of space in macroscopic neural signals.

show abstract

“…Indeed there may exist important relationships between such activity patterns and those that we find in NNs performing TI. It is also worth emphasizing that our findings do not directly address learning processes, for which prior studies have proposed various models and mechanisms [60, 150, 151, 152, 153, 154] (including for explicit variants of TI, where human subjects are informed of the transitive hierarchy [155, 156, 157]). Further, our analyses and neural activity predictions focus on delay period activity, leaving open the question of whether and how neural activity following presentation of both items may contribute to transitive generalization.…”

Section: Discussionmentioning

confidence: 96%

Emergent neural dynamics and geometry for generalization in a transitive inference task

Kay

Wei

Khajeh

et al. 2022

Preprint

View full text Add to dashboard Cite

The ability to make inferences using abstract rules and relations has long been understood to be a hallmark of human intelligence, as evidenced in logic, mathematics, and language. Intriguingly, modern work in animal cognition has established that this ability is evolutionarily widespread, indicating an ancient and possibly foundational role in natural intelligence. Despite this importance, it remains an open question how inference using abstract rules is implemented in the brain - possibly due to a lack of competing hypotheses at the level of collective neural activity and of behavior. Here we report the generation and analysis of a collection of neural networks (NNs) that perform transitive inference (TI), a classical cognitive task that requires inference of a single abstract relation between novel combinations of inputs (if A > B and B > C, then A > C). We found that NNs generated using standard training methods (i) generalize fully (i.e. to all novel combinations of inputs), (ii) generalize when inference requires working memory (WM), a capacity thought to be essential for inference in living subjects, (iii) express multiple emergent behaviors long documented in humans and animals, in addition to novel behaviors not previously studied, and (iv) adopt different solutions that yield alternative predictions for both behavior and collective neural activity. Further, a subset of NNs expressed a "subtractive" solution that was characterized in neural activity space by a simple dynamical pattern (an oscillation) and geometric arrangement (ordered collinearity). Together, these findings show how collective neural activity can accomplish generalization according to an abstract rule, and provide a series of testable hypotheses not previously established in the study of TI. More broadly, these findings suggest new ways to understand how neural systems realize abstract rules and relations.

show abstract

Neural knowledge assembly in humans and deep networks

Cited by 9 publications

References 43 publications

Structuring time: The hippocampus constructs sequence memories that generalize temporal relations across experiences

Structuring time: The hippocampus constructs sequence memories that generalize temporal relations across experiences

Goal-seeking compresses neural codes for space in the human hippocampus and orbitofrontal cortex

Emergent neural dynamics and geometry for generalization in a transitive inference task

Contact Info

Product

Resources

About