Transferring structural knowledge across cognitive maps in humans and models

Mark, Shirley; Moran, Rani; Parr, Thomas; Kennerley, Steven W.; Tej., Behrens

doi:10.1101/860478

Cited by 6 publications

(8 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A truly all-encompassing model of generalization should capture transfer across domains and structural changes. Even though several recent studies have advanced our understanding of how people transfer knowledge across graph structures [ 80 ], state similarities in multi-task reinforcement learning [ 95 ], and target hypotheses supporting generalization [ 90 ], whether or not all of these recruit the same computational principles and neural machinery remains to be seen.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Similarities and differences in spatial and non-spatial cognitive maps

Schulz

Garvert

et al. 2020

PLoS Comput Biol

View full text Add to dashboard Cite

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Previous work has also investigated transfer across domains [80], where inferences about the transition structure in one task can be generalized to other tasks. Whereas we used identical transition structures in both tasks, we nevertheless found asymmetric transfer between domains.…”

Section: Related Workmentioning

confidence: 99%

Similarities and differences in spatial and non-spatial cognitive maps

Schulz

Garvert

et al. 2020

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…This suggests that our hypertransform procedure may rely on a common coding strategy that transforms grid cell activity from one (part of an) environment to another in a predictable manner (e.g. by aligning to a geometric axis in the environment), a possibility supported by recent work 32,33 . Similarly, a related possible cause of a shared representational geometry is input from the head direction (HD) system, which is expected to be "similarly different" across animals for the initial parts of the left and right trials in our data.…”

Section: Discussionmentioning

confidence: 93%

Between-subject prediction reveals a shared representational geometry in the rodent hippocampus

Chen

Manning

Meer

2020

Preprint

View full text Add to dashboard Cite

How a memory system encodes related experiences has consequences for what operations the system supports. For instance, independent coding enables retention of potentially important idiosyncratic details by reducing interference, but makes it difficult to generalize across experiences. Strikingly, the rodent hippocampus constructs statistically independent representations across environments ("global remapping") and assigns individual neuron firing fields to locations within an environment in an apparently random fashion, processes thought to contribute to the role of the hippocampus in episodic memory. This random mapping implies that it should be challenging to predict hippocampal encoding of a given experience in a one subject based on the encoding of that same experience in another subject. Contrary to this prediction, we find that by constructing a common representational space across rats ("hyperalignment"), we can consistently predict data of "right" trials (R) on a T-maze in a target rat based on 1) the "left" trials (L) of the target rat, and 2) the relationship between L and R trials from a different source rat. These cross-subject predictions outperformed a number of control mappings, such as those based on permuted data that broke the relationship between L and R activity for individual neurons, and those based solely on within-subject prediction. This work constitutes proof-of-principle for successful cross-subject prediction of ensemble activity patterns in the hippocampus. This novel approach provides new insights in understanding how different experiences are structured, and suggests further work identifying what aspects of experience encoding are shared vs. unique to an individual.

show abstract

“…In contrast, here we observed that exposure to a single task allowed our participants to improve their knowledge of the task space. Second, while previous studies either focused on learning of multiple simple tasks (Kattner et al, 2017; Schulz et al, 2020), or learning of a single complex graph-like structure (Cleeremans & McClelland, 1991; Garvert et al, 2017; Schapiro et al, 2013), we focused on how humans learn multiple complex structures, an issue which had not been looked at until very recently (Mark et al, 2020; Wu et al, 2019). Third, studies typically focus on the consequences of what is being transferred, for instance, whether there is an immediate benefit to performance or a change in the rate of learning (Braun et al, 2010; Kattner et al, 2017); our paradigm gives us additional insights into the content that is being transferred, specifically, the pool of candidate models that are used for explaining the task increasingly matches the true pool of possible models within the paradigm.…”

Section: Discussionmentioning

confidence: 99%

A mixture of generative models strategy helps humans generalize across tasks

Castañón

Cardoso-Leite

Altarelli

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans extract statistical regularities over time by forming internal representations of the transition probabilities between states. Studies on sequence prediction learning typically focus on how experiencing a particular sequence allows the underlying generative model to be learned. Here we ask what role generative models play when participants experience and learn to predict different sequences from a common generative family. Our novel multi-task prediction paradigm requires participants to complete four sequence learning tasks, each task being a different instantiation of a common generative family. This multi-task environment allows participants to demonstrate two types of learning: (i) within-task learning (i.e., finding the solution to each of the tasks), and (ii) across-task learning (i.e., learning from one task that changes how they approach future tasks). Specifically, to examine across-task learning, we focus on the first four trials of each sequence learning task, as choices during these trials must be driven by participants' background knowledge and not by their in-task learning experience. We show not only that participants arrive with a structured prior, but also that as participants proceed from one task to the next, their prior increasingly resembles the true generative family. Then, to analyse the within-task learning, we model behaviour within each task as arising from a mixture-of-generative-models learning strategy, whereby participants have a pool of candidate explanatory models (mostly within the true generative family) that they continuously update and arbitrate against each other to best explain the stimulus sequence. This framework predicts specific structured errors from participants, as well as a gating mechanism for learning, both of which are observed in the data.

show abstract

Transferring structural knowledge across cognitive maps in humans and models

Cited by 6 publications

References 39 publications

Similarities and differences in spatial and non-spatial cognitive maps

Similarities and differences in spatial and non-spatial cognitive maps

Between-subject prediction reveals a shared representational geometry in the rodent hippocampus

A mixture of generative models strategy helps humans generalize across tasks

Contact Info

Product

Resources

About