Modeling memory: what do we learn from attractor neural networks?

Brunel, Nicolas; Nadal, Jean-Pierre

doi:10.1016/s0764-4469(97)89830-7

Cited by 6 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, Hebb's famous postulate (Hebb, 1949) that causally correlated firing of connected neurons could lead to a strengthening of the connection, was based on the suggestion that the correlated firing would be maintained in a recurrently connected cell assembly beyond the time of a transient stimulus (Hebb, 1949). Since then, analytic and computational models have demonstrated the ability of such recurrent networks to produce multiple discrete attractor states (Brunel and Nadal, 1998), as in Hopfield networks (Hopfield, 1982, 1984), or to be capable of integration over time via a marginally stable network, often termed a line attractor (Zhang, 1996; Compte et al, 2000). Much of the work on these systems has assumed either static synapses, or considered changes in synaptic strength via long-term plasticity occurring on a much slower timescale than the dynamics of neuronal responses.…”

Section: Introductionmentioning

confidence: 99%

Stimulus number, duration and intensity encoding in randomly connected attractor networks with synaptic depression

Miller

2013

Front. Comput. Neurosci.

View full text Add to dashboard Cite

Randomly connected recurrent networks of excitatory groups of neurons can possess a multitude of attractor states. When the internal excitatory synapses of these networks are depressing, the attractor states can be destabilized with increasing input. This leads to an itinerancy, where with either repeated transient stimuli, or increasing duration of a single stimulus, the network activity advances through sequences of attractor states. We find that the resulting network state, which persists beyond stimulus offset, can encode the number of stimuli presented via a distributed representation of neural activity with non-monotonic tuning curves for most neurons. Increased duration of a single stimulus is encoded via different distributed representations, so unlike an integrator, the network distinguishes separate successive presentations of a short stimulus from a single presentation of a longer stimulus with equal total duration. Moreover, different amplitudes of stimulus cause new, distinct activity patterns, such that changes in stimulus number, duration and amplitude can be distinguished from each other. These properties of the network depend on dynamic depressing synapses, as they disappear if synapses are static. Thus, short-term synaptic depression allows a network to store separately the different dynamic properties of a spatially constant stimulus.

show abstract

Section: Introductionmentioning

confidence: 99%

Stimulus number, duration and intensity encoding in randomly connected attractor networks with synaptic depression

Miller

2013

Front. Comput. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Networks of recurrently connected units are capable of producing a diversity of distinct pointattractor states [1]. The component units of these networks can be single neurons, or groups of correlated, similarly responsive neurons.…”

Section: Circuits With Multiple Point-attractor Statesmentioning

confidence: 99%

Spatiotemporal discrimination in attractor networks with short-term synaptic plasticity

Shlaer

Ballintyn

Miller

2018

Preprint

View full text Add to dashboard Cite

¶ These authors contributed equally to this work. AbstractWe demonstrate the ability of a randomly connected attractor network with dynamic synapses to discriminate between similar sequences containing multiple stimuli and suggest such networks provide a general basis for neural computations in the brain. The network is based on units representing assemblies of pools of neurons, with preferentially strong recurrent excitatory connections within each unit. Such excitatory feedback to a unit can generate bistability, though in many networks only under conditions of net excitatory input from other units. Weak interactions between units leads to a multiplicity of attractor states, within which information can persist beyond stimulus offset. When a new stimulus arrives, the prior state of the network impacts the encoding of the incoming information, with short-term synaptic depression ensuring an itinerancy between sets of active units. We assess the ability of such a network to encode the identity of sequences of stimuli, so as to provide a template for sequence recall, or decisions based on accumulation of evidence. Across a range of parameters, such networks produce the primacy (better final encoding of the earliest stimuli) and recency (better final encoding of the latest stimuli) observed in human recall data and can retain the information needed to make a binary choice based on total number of presentations of a specific stimulus. Similarities and differences in the final states of the network produced by different sequences lead to predictions of specific errors that could arise when an animal or human subject generalizes from training data, when the training data comprises a subset of the entire stimulus repertoire. We suggest that such networks can provide the robust general purpose computational engines needed for us to solve many cognitive tasks.

show abstract

“…A number of workers have used artificial associative memories based upon the Hopfield model in their attempts to explain the mechanisms underlying the operation both of normal and abnormal human memory, including phenomena such as pseudo-rehearsal and unlearning [21,32]. Hopfield-style networks have also been used in attempts to simulate the behaviour of human memory when subjected to various types of damage, in order to elicit information about the underlying causes of diseases such as Alzheimer's dementia and schizophrenia [8,22,[33][34][35].…”

Section: Biological Plausibilitymentioning

confidence: 99%

High capacity recurrent associative memories

2004

View full text Add to dashboard Cite

Various algorithms for constructing weight matrices for Hopfield-type associative memories are reviewed, including ones with much higher capacity than the basic model. These alternative algorithms either iteratively approximate the projection weight matrix or use simple perceptron learning.An experimental investigation of the performance of networks trained by these algorithms is presented, including measurements of capacity, training time and their ability to correct corrupted versions of the training patterns.

show abstract

Modeling memory: what do we learn from attractor neural networks?

Cited by 6 publications

References 16 publications

Stimulus number, duration and intensity encoding in randomly connected attractor networks with synaptic depression

Stimulus number, duration and intensity encoding in randomly connected attractor networks with synaptic depression

Spatiotemporal discrimination in attractor networks with short-term synaptic plasticity

High capacity recurrent associative memories

Contact Info

Product

Resources

About