Forgetting Leads to Chaos in Attractor Networks

Pereira-Obilinovic, Ulises; Aljadeff, Johnatan; Brunel, Nicolas

doi:10.1103/physrevx.13.011009

Cited by 14 publications

(13 citation statements)

References 74 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…p is defined as the maximal number of patterns that are still retrievable or, equivalently, the maximal age at which patterns can still be retrieved. This scenario is similar to previously studied attractor neural network models with online learning and decay term in the synaptic updates [ 27 , 31 ], or networks with binary synapses [ 18 , 32 , 33 ] in which memories are forgotten exponentially.…”

Section: Modelsupporting

confidence: 62%

“…As demonstrated in Fig 7 , the storage capacity of the network increases logarithmically with the network size N , with a prefactor that is proportional to τ = 1/(2 r 1 ) as indicated by Eq (10) . This case is similar to the model with continuous synapses and exponentially decaying memories, as discussed in [ 27 , 28 , 31 ].…”

Section: Resultsmentioning

confidence: 82%

“…Here, we focus on the standard coding (‘balanced input’) scenario f = 0.5, and θ = 0. Simulations show that sometimes, when initialized at a pattern stored in the past, the network mistakenly retrieves the most recently shown pattern η u instead [ 31 ]. To calculate the storage capacity, we thus need to define two overlaps: where represents the fixed point of Eq (1) with initial condition , and η u represent the most recently shown pattern.…”

Section: Methodsmentioning

confidence: 99%

See 2 more Smart Citations

Attractor neural networks with double well synapses

Feng,

Brunel

2024

PLoS Comput Biol

Self Cite

View full text Add to dashboard Cite

It is widely believed that memory storage depends on activity-dependent synaptic modifications. Classical studies of learning and memory in neural networks describe synaptic efficacy either as continuous or discrete. However, recent results suggest an intermediate scenario in which synaptic efficacy can be described by a continuous variable, but whose distribution is peaked around a small set of discrete values. Motivated by these results, we explored a model in which each synapse is described by a continuous variable that evolves in a potential with multiple minima. External inputs to the network can switch synapses from one potential well to another. Our analytical and numerical results show that this model can interpolate between models with discrete synapses which correspond to the deep potential limit, and models in which synapses evolve in a single quadratic potential. We find that the storage capacity of the network with double well synapses exhibits a power law dependence on the network size, rather than the logarithmic dependence observed in models with single well synapses. In addition, synapses with deeper potential wells lead to more robust information storage in the presence of noise. When memories are sparsely encoded, the scaling of the capacity with network size is similar to previously studied network models in the sparse coding limit.

show abstract

Section: Modelsupporting

confidence: 62%

Section: Resultsmentioning

confidence: 82%

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Attractor neural networks with double well synapses

Feng,

Brunel

2024

PLoS Comput Biol

Self Cite

View full text Add to dashboard Cite

show abstract

“…Note that this plasticity rule is similar to previously studied 'palimpsest' schemes, see e.g. (Mézard et al, 1986;Pereira-Obilinovic et al, 2023). For simplicity, we assume that the rates r µ i used to compute synaptic changes during presentation of an image µ are entirely determined by external inputs, r µ i = ϕ(ξ µ i ) for all i when image µ is presented.…”

Section: Assumptionsmentioning

confidence: 91%

Visual familiarity learning at multiple timescales in the primate inferotemporal cortex

Mohan,

Pereira-Obilinovic,

Srednyak

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Humans and other primates can rapidly detect familiar objects and distinguish them from never-before-seen novel objects. We have an astonishing capacity to remember the details of visual scenes even after a single, fleeting experience. This ability is thought to rely in part on experience-dependent changes in the primate inferotemporal cortex (IT). Single neurons in IT encode visual familiarity by discriminating between novel and familiar stimuli, with stronger neural activity on average for novel images. However, key open questions are to understand how neural encoding in IT changes as images progress from novel to highly familiar, and what learning rules and computations can account for learning-dependent changes in IT activity. Here, we investigate the timescales over which novel stimuli become familiar by recording in IT as initially novel images become increasingly familiar both within and across days. We identified salient and persistent memory-related signals in IT that spanned multiple timescales of minutes, hours, and days. Average neural activity progressively decreased with familiarity as firing rates were strongest for novel, weaker for intermediately familiar, and weakest for highly familiar images. Neural signatures of familiarity learning were slow to develop as response reductions to initially-novel images emerged gradually over multiple days (or hundreds of views) of visual experience. In addition to slow changes that emerged across sessions, neural responses to novel images showed rapid decreases with familiarity within single sessions. To gain insight into the mechanisms underlying changes of visual responses with familiarity, we use computational modeling to investigate which plasticity rules are consistent with these changes. Learning rules inferred from the neural data revealed a strong diversity with many neurons following a ‘negative’ plasticity rule as they exhibited synaptic depression over the course of learning across multiple days. A recurrent network model with two plasticity time constants – a slow time constant for long timescales and a fast time constant for short timescales – captured key dynamic features accompanying the transition from novel to familiar, including a gradual decrease in firing rates over multiple sessions, and a rapid decrease in firing rates within single sessions. Our findings suggest that distinct and complementary plasticity rules operating at different timescales may underlie the inferotemporal code for visual familiarity.

show abstract

“…T heoretical constructs called attractor networks provide a model for memory in the brain. A new study of such networks traces the route by which memories are stored and ultimately forgotten [1]. The mathematical model and simulations show that, as they age, memories recorded in patterns of neural activity become chaotic-impossible to predict-before disintegrating into random noise.…”

mentioning

confidence: 99%

Memories Become Chaotic before They Are Forgotten

Ball¹

2023

Physics

View full text Add to dashboard Cite

Forgetting Leads to Chaos in Attractor Networks

Cited by 14 publications

References 74 publications

Attractor neural networks with double well synapses

Attractor neural networks with double well synapses

Visual familiarity learning at multiple timescales in the primate inferotemporal cortex

Memories Become Chaotic before They Are Forgotten

Contact Info

Product

Resources

About