Codimension-2 parameter space structure of continuous-time recurrent neural networks

Beer, Randall D.

doi:10.1007/s00422-022-00938-5

Cited by 4 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We found that multitasking networks sometimes employed parameter bifurcations across input conditions for dissimilar computations. The relationship between high dimensional parameter bifurcations, composition and computation is an important area of future research; see for example recent work [40][41][42][43] .…”

Section: Discussionmentioning

confidence: 99%

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Driscoll

Shenoy

Sussillo

2022

Preprint

View full text Add to dashboard Cite

Flexible computation is a hallmark of intelligent behavior. Yet, little is known about how neural networks contextually reconfigure for different computations. Humans are able to perform a new task without extensive training, presumably through the composition of elementary processes that were previously learned. Cognitive scientists have long hypothesized the possibility of a compositional neural code, where complex neural computations are made up of constituent components; however, the neural substrate underlying this structure remains elusive in biological and artificial neural networks. Here we identified an algorithmic neural substrate for compositional computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses of networks revealed learned computational strategies that mirrored the modular subtask structure of the task-set used for training. Dynamical motifs such as attractors, decision boundaries and rotations were reused across different task computations. For example, tasks that required memory of a continuous circular variable repurposed the same ring attractor. We show that dynamical motifs are implemented by clusters of units and are reused across different contexts, allowing for flexibility and generalization of previously learned computation. Lesioning these clusters resulted in modular effects on network performance: a lesion that destroyed one dynamical motif only minimally perturbed the structure of other dynamical motifs. Finally, modular dynamical motifs could be reconfigured for fast transfer learning. After slow initial learning of dynamical motifs, a subsequent faster stage of learning reconfigured motifs to perform novel tasks. This work contributes to a more fundamental understanding of compositional computation underlying flexible general intelligence in neural systems. We present a conceptual framework that establishes dynamical motifs as a fundamental unit of computation, intermediate between the neuron and the network. As more whole brain imaging studies record neural activity from multiple specialized systems simultaneously, the framework of dynamical motifs will guide questions about specialization and generalization across brain regions.

show abstract

Section: Discussionmentioning

confidence: 99%

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Driscoll

Shenoy

Sussillo

2022

Preprint

View full text Add to dashboard Cite

show abstract

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Driscoll,

Shenoy,

Sussillo

2024

Nat Neurosci

View full text Add to dashboard Cite

Flexible computation is a hallmark of intelligent behavior. However, little is known about how neural networks contextually reconfigure for different computations. In the present work, we identified an algorithmic neural substrate for modular computation through the study of multitasking artificial recurrent neural networks. Dynamical systems analyses revealed learned computational strategies mirroring the modular subtask structure of the training task set. Dynamical motifs, which are recurring patterns of neural activity that implement specific computations through dynamics, such as attractors, decision boundaries and rotations, were reused across tasks. For example, tasks requiring memory of a continuous circular variable repurposed the same ring attractor. We showed that dynamical motifs were implemented by clusters of units when the unit activation function was restricted to be positive. Cluster lesions caused modular performance deficits. Motifs were reconfigured for fast transfer learning after an initial phase of learning. This work establishes dynamical motifs as a fundamental unit of compositional computation, intermediate between neuron and network. As whole-brain studies simultaneously record activity from multiple specialized systems, the dynamical motif framework will guide questions about specialization and generalization.

show abstract

Seeing double with a multifunctional reservoir computer

Flynn,

Tsachouridis,

Amann

2023

Chaos: An Interdisciplinary Journal of Nonlinear Science

View full text Add to dashboard Cite

Multifunctional biological neural networks exploit multistability in order to perform multiple tasks without changing any network properties. Enabling artificial neural networks (ANNs) to obtain certain multistabilities in order to perform several tasks, where each task is related to a particular attractor in the network’s state space, naturally has many benefits from a machine learning perspective. Given the association to multistability, in this paper, we explore how the relationship between different attractors influences the ability of a reservoir computer (RC), which is a dynamical system in the form of an ANN, to achieve multifunctionality. We construct the “seeing double” problem in order to systematically study how a RC reconstructs a coexistence of attractors when there is an overlap between them. As the amount of overlap increases, we discover that for multifunctionality to occur, there is a critical dependence on a suitable choice of the spectral radius for the RC’s internal network connections. A bifurcation analysis reveals how multifunctionality emerges and is destroyed as the RC enters a chaotic regime that can lead to chaotic itinerancy.

show abstract

Codimension-2 parameter space structure of continuous-time recurrent neural networks

Cited by 4 publications

References 39 publications

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

Seeing double with a multifunctional reservoir computer

Contact Info

Product

Resources

About