High capacity and dynamic accessibility in associative memory networks with context-dependent neuronal and synaptic gating

Abstract

Context, such as behavioral state, is known to modulate memory formation and retrieval, but is usually ignored in associative memory models. Here, we propose several types of contextual modulation for associative memory networks that greatly increase their performance. In these networks, context inactivates specific neurons and connections, which modulates the effective connectivity of the network. Memories are stored only by the active components, thereby reducing interference from memories acquired in other contexts. Such networks exhibit several beneficial characteristics, including enhanced memory capacity, high robustness to noise, increased robustness to memory overloading, and better memory retention during continual learning. Furthermore, memories can be biased to have different relative strengths, or even gated on or off, according to contextual cues, providing a candidate model for cognitive control of memory and efficient memory search. An external context-encoding network can dynamically switch the memory network to a desired state, which we liken to experimentally observed contextual signals in prefrontal cortex and hippocampus. Overall, our work illustrates the benefits of organizing memory around context, and provides an important link between behavioral studies of memory and mechanistic details of neural circuits. SIGNIFICANCEMemory is context dependent -both encoding and recall vary in effectiveness and speed depending on factors like location and brain state during a task. We apply this idea to a simple computational model of associative memory through contextual gating of neurons and synaptic connections. Intriguingly, this results in several advantages, including vastly enhanced memory capacity, better robustness, and flexible memory gating. Our model helps to explain (i) how gating and inhibition contribute to memory processes, (ii) how memory access dynamically changes over time, and (iii) how context representations, such as those observed in hippocampus and prefrontal cortex, may interact with and control memory processes.FIG. 1. Schematic of the context-modular memory network. A, Associative memory is defined hierarchically through a set of contexts (c 1 to c 5 ) and memory patterns (m 1 to m 15 ) assigned to each one. B, Network implementation: neurons are arranged into contextual configurations (subnetworks) in two ways: neuron-specific gating, where context is defined as a proportion of available neurons (colored rings; defined randomly, spatial localization is illustrative), and synapse-specific gating, where context is defined as a proportion of gated synapses (red cross, bottom right inset). Context is controlled by an external contextencoding network, such that one context is active at a time (black ring), and memories outside of the active context remain dormant. C, Contextual configurations change the effective connectivity matrix of the associative memory network: neuron-specific gating removes particular columns and rows (left), synapse-specific gating removes individual elemen...

Formation and computational implications of assemblies in neural circuits

“…The abstractions in these models have enabled extensive theoretical results related to these attractor networks, including the theoretical limit for the number of patterns or concepts stored in a recurrent network (Amit et al, 1985;Gardner, 1988). Various aspects in the models have also been made more biologically realistic, for example sparse coding (Amari, 1989;Gripon et al, 2016), asymmetric connections (Tsodyks, 1989) or context dependency (Podlaski et al, 2020). It is interesting to compare the abstracted learning rules of associative memory models with their more biologically plausible local and dynamic learning counterparts, as they are used in studies on assembly formation.…”

When shared concept cells support associations: Theory of overlapping memory engrams

“…Even the co-activation of C1 and P0, is not enough, to automatically activate P1 on its own, in the context of the static model without adaptation, or inhibition. In our framework the activation of one context favors the recall of concepts associated to it, and it can be qualitative compared to neuron-specific gating model proposed in [ 30 ], where the activation of one context defines a subset of available neurons.…”

The Computational and Neural Bases of Context-Dependent Learning

“…A powerful solution is to create separate networks (Aljundi et al 2017), or subnetworks (Rusu et al 2016, Li & Hoiem 2017, for each task (context) and use an input representing the current context (task identity) to index the appropriate (sub)network that needs to be called upon. To avoid having to grow the network with time, which may limit generalization across tasks and require biologically implausible mechanisms, fixed network architectures with context-dependent gating have also been studied (Masse et al 2018, Podlaski et al 2020, Flesch et al 2022. In these networks, a fraction of the units is used for each task (while the rest are gated by setting their activities to 0).…”