Neural representation of abstract task structure during generalization

Vaidya, Avinash R.; Jones, Henry M; Castillo, Johanny; Badre, David

doi:10.1101/2020.07.21.213009

Cited by 3 publications

(3 citation statements)

References 71 publications

(89 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Understanding how the brain represents abstract task state-spaces remains a major challenge. Our data join growing evidence suggesting state-space representations rely on the hippocampal formation ( Garvert et al., 2017 ; Miller et al., 2017 ; Stachenfeld et al., 2017 ; Vikbladh et al., 2019 ; Wang et al., 2020 ; Zhou et al., 2019b ) and interconnected regions in the ventral PFC ( Schuck et al., 2016 ; Vaidya et al., 2020 ; Wang et al., 2020 ; Wilson et al., 2014 ; Zhou et al., 2019a , 2019b ). This is particularly interesting in light of the historical role of these regions in generalization and relational reasoning ( Barron et al., 2013 ; Bowman and Zeithamova, 2018 ; Cohen and Eichenbaum, 1993 ; Morrissey et al., 2017 ; Preston and Eichenbaum, 2013 ; Zeithamova and Preston, 2010 ), which are essential for efficient task representations.…”

Section: Discussionsupporting

confidence: 84%

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Baram

Muller

Nili

et al. 2021

Neuron

View full text Add to dashboard Cite

Summary Knowledge of the structure of a problem, such as relationships between stimuli, enables rapid learning and flexible inference. Humans and other animals can abstract this structural knowledge and generalize it to solve new problems. For example, in spatial reasoning, shortest-path inferences are immediate in new environments. Spatial structural transfer is mediated by cells in entorhinal and (in humans) medial prefrontal cortices, which maintain their co-activation structure across different environments and behavioral states. Here, using fMRI, we show that entorhinal and ventromedial prefrontal cortex (vmPFC) representations perform a much broader role in generalizing the structure of problems. We introduce a task-remapping paradigm, where subjects solve multiple reinforcement learning (RL) problems differing in structural or sensory properties. We show that, as with space, entorhinal representations are preserved across different RL problems only if task structure is preserved. In vmPFC and ventral striatum, representations of prediction error also depend on task structure.

show abstract

Section: Discussionsupporting

confidence: 84%

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Baram

Muller

Nili

et al. 2021

Neuron

View full text Add to dashboard Cite

show abstract

“…If this is the case, value can be seen as one important aspect or derivative of such a representation, particularly in the experimental settings typically employed in behavioral neuroscience studies. This hypothesis has considerable support from both older work as well as recent correlative studies in humans [39][40][41][42][43][44], non-human primates [45], and rodents [31,[46][47][48][49][50][51].…”

Section: Possibility Of Value As Part Of Task Structure Representationmentioning

confidence: 79%

Is the core function of orbitofrontal cortex to signal values or make predictions?

Zhou

Gardner

Schoenbaum

2021

Current Opinion in Behavioral Sciences

View full text Add to dashboard Cite

One dominant hypothesis about the function of the orbitofrontal cortex (OFC) is that the OFC signals the subjective values of possible outcomes to other brain areas for learning and decision making. This popular view generally neglects the fact that OFC is not necessary for simple value-based behavior (i.e., when values have been directly experienced). An alternative, emerging view suggests that OFC plays a more general role in representing structural information about the task or environment, derived from prior experience, and relevant to predicting behavioral outcomes, such as value. From this perspective, value signaling is simply one derivative of the core underlying function of OFC. New data in favor of both views have been accumulating rapidly. Here we review these new data in discussing the relative merits of these two ideas.

show abstract

“…Previous research (Baxter, 2019; Franklin & Frank, 2018; Musslick et al, 2017; Vaidya, Jones, Castillo, & Badre, 2020; Zambaldi et al, 2018) indeed illustrated that sharing task representations significantly improves learning and generalization across tasks, two hallmarks of human flexibility. However, sharing task representations in a neural network severely impacts the network’s ability to perform more than one task at the same time (i.e., to multi-task; Alon et al, 2017; Musslick et al, 2017; Musslick, Saxe, Novick, Reichman, & Cohen, 2020).…”

Section: Introductionmentioning

confidence: 96%

Using top-down modulation to optimally balance shared versus separated task representations

Verbeke

Verguts

2021

Preprint

View full text Add to dashboard Cite

Human adaptive behavior requires continually learning and performing a wide variety of tasks, often with very little practice. To accomplish this, it is crucial to separate neural representations of different tasks in order to avoid interference. At the same time, sharing neural representations supports generalization and allows faster learning. Therefore, a crucial challenge is to find an optimal balance between shared versus separated representations. Typically, models of human cognition employ top-down gating signals to separate task representations, but there exist surprisingly little systematic computational investigations of how such gating is best implemented. We identify and systematically evaluate two crucial features of gating signals. First, top-down input can be processed in an additive or multiplicative manner. Second, the gating signals can be adaptive (learned) or non-adaptive (random). We cross these two features, resulting in four gating models which are tested on a variety of input datasets and tasks with different degrees of stimulus-action mapping overlap. The multiplicative adaptive gating model outperforms all other models in terms of accuracy. Moreover, this model develops hidden units that optimally share representations between tasks. Specifically, different than the binary approach of currently popular latent state models, it exploits partial overlap between tasks.

show abstract

Neural representation of abstract task structure during generalization

Cited by 3 publications

References 71 publications

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Entorhinal and ventromedial prefrontal cortices abstract and generalize the structure of reinforcement learning problems

Is the core function of orbitofrontal cortex to signal values or make predictions?

Using top-down modulation to optimally balance shared versus separated task representations

Contact Info

Product

Resources

About