Slow synaptic dynamics in a network: From exponential to power-law forgetting

Abstract

We investigate a mean-field model of interacting synapses on a directed neural network. Our interest lies in the slow adaptive dynamics of synapses, which are driven by the fast dynamics of the neurons they connect. Cooperation is modelled from the usual Hebbian perspective, while competition is modelled by an original polarity-driven rule. The emergence of a critical manifold culminating in a tricritical point is crucially dependent on the presence of synaptic competition. This leads to a universal 1/t power-law relaxation of the mean synaptic strength along the critical manifold and an equally universal 1/ √ t relaxation at the tricritical point, to be contrasted with the exponential relaxation that is otherwise generic. In turn, this leads to the natural emergence of long-and short-term memory from different parts of parameter space in a synaptic network, which is the most novel and important result of our present investigations.

Storing and retrieving long-term memories: cooperation and competition in synaptic dynamics

“…the palimpsest effect [28,29]). This is indeed what is done in the model network of synapses and neurons [106] that we will describe in the following. Like the Fusi [79] model, it is a model of discrete rather than continuous synapses; unlike it, however, here, there are explicit mechanisms of synaptic weight change via mechanisms of competing and cooperating synapses that depend intimately on neuronal firing rates.…”

“…There is a left critical case, where J 1 = J 3 = J (L) c , while J 2 remains noncritical, and a right one, where J 2 = J 3 = J (R) c , while J 1 remains non-critical. The critical synaptic strength obeys J c > 1 3 [106]. We thus conclude that the critical point is always strengthening, as J c is always larger then the 'natural' initial value J(0) = 0, corresponding to a random mixture of strong and weak synapses in equal proportions.…”

“…where τ 0 and J 0 are obtainable in terms of the model parameters [106]. In Regime II (see Figure 6, right), there are two attractive (stable) fixed points at J 1 and J 2 , and an intermediate repulsive (unstable) one at J 3 .…”

“…A typical example arises in, say, the context of financial markets; trading rules between different agents at an individual level can result in specific sets of traders, or their representative strategies, winning over their competitors. This makes for interesting analogies with competitive learning; approaches based on this have therefore successfully been used to investigate a wide variety of topics, ranging from the diffusion of innovations [101,102] through gap junction connectivity in the pancreas [103] to the dynamics of competing synapses [104][105][106]. It is the latter which will concern us here, but in the interests of completeness, we first briefly review an agent-based model of competitive learning in the following [101].…”

“…The two possible kinds of critical dynamical behaviour: left critical case (J 1 = J 3 = J (L) c ) and right critical case (J 2 = J 3 = J (R) c) (after Ref [106]…”