Temporal chunking as a mechanism for unsupervised learning of task-sets

Bouchacourt, Flora; Palminteri, Stefano; Koechlin, Etienne; Ostojic, Srdjan

doi:10.7554/elife.50469

Cited by 20 publications

(11 citation statements)

References 106 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hierarchies exist beyond the somewhat simple learning of compositional sequences, and it is expected that hierarchical models share common basic features despite solving highly distinct problems. For instance, a recent example of a hierarchical model for working memory uses two different networks: an associative network and a task-set network [ 56 ]. In our setting, the associative network could be identified with the motifs (fast clock+read-out) whereas the task-set network would correspond to the syntax (slow clock+interneurons).…”

Section: Discussionmentioning

confidence: 99%

Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

2021

View full text Add to dashboard Cite

Sequential behaviour is often compositional and organised across multiple time scales: a set of individual elements developing on short time scales (motifs) are combined to form longer functional sequences (syntax). Such organisation leads to a natural hierarchy that can be used advantageously for learning, since the motifs and the syntax can be acquired independently. Despite mounting experimental evidence for hierarchical structures in neuroscience, models for temporal learning based on neuronal networks have mostly focused on serial methods. Here, we introduce a network model of spiking neurons with a hierarchical organisation aimed at sequence learning on multiple time scales. Using biophysically motivated neuron dynamics and local plasticity rules, the model can learn motifs and syntax independently. Furthermore, the model can relearn sequences efficiently and store multiple sequences. Compared to serial learning, the hierarchical model displays faster learning, more flexible relearning, increased capacity, and higher robustness to perturbations. The hierarchical model redistributes the variability: it achieves high motif fidelity at the cost of higher variability in the between-motif timings.

show abstract

Section: Discussionmentioning

confidence: 99%

Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

2021

View full text Add to dashboard Cite

show abstract

“…Another intriguing possibility is that unsupervised processes facilitate continual learning in biological systems by clustering neural representations according to their context. Hebbian learning might encourage the formation of orthogonal neural codes for different temporally contexts 147 , which in turn allows tasks to be learned in different neural subspaces 146 . The curious phenomenon of "representational drift" (where neural codes meander unpredictably over time) 148 might reflect the allocation of information to different neural circuits in distinct contexts, allowing for task knowledge to be partitioned in a way that minimises interference 149 .…”

Section: Neural Resource Allocation During Task Learningmentioning

confidence: 99%

If deep learning is the answer, what is the question?

2020

View full text Add to dashboard Cite

Neuroscience research is undergoing a minor revolution. Recent advances in machine learning and artificial intelligence (AI) research have opened up new ways of thinking about neural computation. Many researchers are excited by the possibility that deep neural networks may offer theories of perception, cognition and action for biological brains. This perspective has the potential to radically reshape our approach to understanding neural systems, because the computations performed by deep networks are learned from experience, not endowed by the researcher. If so, how can neuroscientists use deep networks to model and understand biological brains? What is the outlook for neuroscientists who seek to characterise computations or neural codes, or who wish to understand perception, attention, memory, and executive functions? In this Perspective, our goal is to offer a roadmap for systems neuroscience research in the age of deep learning. We discuss the conceptual and methodological challenges of comparing behaviour, learning dynamics, and neural representation in artificial and biological systems. We highlight new research questions that have emerged for neuroscience as a direct consequence of recent advances in machine learning.

show abstract

“…However, brains are able to perform multiple tasks in the same environment. Those tasks often involve sequential behaviour at multiple timescales ( Bouchacourt et al, 2019 ). Pursuing goals sometimes requires following a sequence of subroutines, with short-term/interim objectives, themselves divided into elemental skills ( Botvinick et al, 2019 ).…”

Section: A Model-free Agent Using An Actor–critic Architecturementioning

confidence: 99%

Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation

Tessereau

O’Dea

Coombes

et al. 2021

Brain and Neuroscience Advances

View full text Add to dashboard Cite

Humans and non-human animals show great flexibility in spatial navigation, including the ability to return to specific locations based on as few as one single experience. To study spatial navigation in the laboratory, watermaze tasks, in which rats have to find a hidden platform in a pool of cloudy water surrounded by spatial cues, have long been used. Analogous tasks have been developed for human participants using virtual environments. Spatial learning in the watermaze is facilitated by the hippocampus. In particular, rapid, one-trial, allocentric place learning, as measured in the delayed-matching-to-place variant of the watermaze task, which requires rodents to learn repeatedly new locations in a familiar environment, is hippocampal dependent. In this article, we review some computational principles, embedded within a reinforcement learning framework, that utilise hippocampal spatial representations for navigation in watermaze tasks. We consider which key elements underlie their efficacy, and discuss their limitations in accounting for hippocampus-dependent navigation, both in terms of behavioural performance (i.e. how well do they reproduce behavioural measures of rapid place learning) and neurobiological realism (i.e. how well do they map to neurobiological substrates involved in rapid place learning). We discuss how an actor–critic architecture, enabling simultaneous assessment of the value of the current location and of the optimal direction to follow, can reproduce one-trial place learning performance as shown on watermaze and virtual delayed-matching-to-place tasks by rats and humans, respectively, if complemented with map-like place representations. The contribution of actor–critic mechanisms to delayed-matching-to-place performance is consistent with neurobiological findings implicating the striatum and hippocampo-striatal interaction in delayed-matching-to-place performance, given that the striatum has been associated with actor–critic mechanisms. Moreover, we illustrate that hierarchical computations embedded within an actor–critic architecture may help to account for aspects of flexible spatial navigation. The hierarchical reinforcement learning approach separates trajectory control via a temporal-difference error from goal selection via a goal prediction error and may account for flexible, trial-specific, navigation to familiar goal locations, as required in some arm-maze place memory tasks, although it does not capture one-trial learning of new goal locations, as observed in open field, including watermaze and virtual, delayed-matching-to-place tasks. Future models of one-shot learning of new goal locations, as observed on delayed-matching-to-place tasks, should incorporate hippocampal plasticity mechanisms that integrate new goal information with allocentric place representation, as such mechanisms are supported by substantial empirical evidence.

show abstract

Temporal chunking as a mechanism for unsupervised learning of task-sets

Cited by 20 publications

References 106 publications

Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

Learning compositional sequences with multiple time scales through a hierarchical network of spiking neurons

If deep learning is the answer, what is the question?

Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation

Contact Info

Product

Resources

About