Early selection of task-relevant features through population gating

Barbosa, Joao; Proville, Rémi; Rodgers, Chris C.; DeWeese, Michael R.; Ostojic, Srdjan; Boubenec, Yves

doi:10.1038/s41467-023-42519-5

Cited by 14 publications

(10 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Given the that the phase-coding solution matches the phenomenon we set out to investigate, whereas the rate-coding solution shows similarities to previous work on fixed point dynamics in RNNs [ 26 , 30 , 31 ], we focus here on the analysis of the ‘phase-coding’ solution ( Fig 2A, 2B and 2C ). The rate-coding and its similarities to existing work is detailed in the Supplementary ( S4 Fig ).…”

Section: Resultsmentioning

confidence: 98%

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals,

Macke,

Barak

2024

PLoS Comput Biol

View full text Add to dashboard Cite

Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

show abstract

Section: Resultsmentioning

confidence: 98%

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals,

Macke,

Barak

2024

PLoS Comput Biol

View full text Add to dashboard Cite

show abstract

“…Given the that the phase-coding solution matches the phenomenon we set out to investigate, whereas the rate-coding solution shows similarities to previous work on fixed point dynamics in RNNs [26, 30, 31], we focus here on the analysis of the ‘phase-coding’ solution (Fig 2A-C). The rate-coding and its similarities to existing work is detailed in the Supplementary (S4 Fig).…”

Section: Resultsmentioning

confidence: 99%

“…We used trained recurrent neural networks [20, 22, 24]. RNNs can be trained to reproduce both neural activity and behavior [17, 23, 31, 34, 36]. The resulting networks can be understood in terms of their dynamical objects, i.e.…”

Section: Discussionmentioning

confidence: 99%

“…These units either directly transmit the input oscillation, creating an in-phase oscillation, or flip its sign to create an anti-phase oscillation. In κ space, activity lies on two non-overlapping cycles Given the that the phase-coding solution matches the phenomenon we set out to investigate, whereas the rate-coding solution shows similarities to previous work on fixed point dynamics in RNNs [26,30,31], we focus here on the analysis of the 'phase-coding' solution (Fig 2A-C). The rate-coding and its similarities to existing work is detailed in the Supplementary (S4 Fig) . Single trials have a limited duration, and hence the cycles we observed might arise either from a transient dynamical phenomenon or from stable attractors.…”

Section: Phase-coded Memories Correspond To Limit Cycle Attractorsmentioning

confidence: 99%

“…RNNs are universal dynamical systems, in the sense that they can approximate finite time trajectories of any dynamical system [29,30]. We are thus able to use these networks as hypothesis generators -first training them on a cognitive task, and then reverse engineering the resulting networks [17,20,24,26,[31][32][33][34][35][36][37][38].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Pals

Macke

Barak

2023

Preprint

View full text Add to dashboard Cite

Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or 'frame of reference'. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.

show abstract

Biologically plausible models of cognitive flexibility: merging recurrent neural networks with full-brain dynamics

van Holk,

Mejias

2024

Current Opinion in Behavioral Sciences

View full text Add to dashboard Cite

Early selection of task-relevant features through population gating

Cited by 14 publications

References 55 publications

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Trained recurrent neural networks develop phase-locked limit cycles in a working memory task

Biologically plausible models of cognitive flexibility: merging recurrent neural networks with full-brain dynamics

Contact Info

Product

Resources

About