Slowly evolving connectivity in recurrent neural networks: I. The extreme dilution regime

Abstract

We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain phase diagrams upon making the condensed ansatz (i.e. recall of one pattern). These show that, as the connectivity temperature is lowered, the volume of the retrieval phase diverges and the fraction of mis-aligned spins is reduced. Still one always retains a region in the retrieval phase where recall states other than the one corresponding to the 'condensed' pattern are locally stable, so the associative memory character of our model is preserved.

Slowly evolving random graphs II: adaptive geometry in finite-connectivity Hopfield models

“…To get a better understanding of this effect, we also compute the fraction of misaligned spins and the fraction of sites with vanishing local fields as functions of the ratio of the characteristic temperatures of the two dynamic processes. In the limit of extreme dilution our expressions reproduce the results of [26].…”

“…The present model is an extension of the one presented in [26] where geometry was of an adaptive nature. There, one could make important analytic simplifications owing to the choice of 'extreme dilution' scaling whereby each node in the graph was connected on average to a vanishing fraction of other nodes although this fraction still contained an infinite number of nodes.…”

Dynamic rewiring in small world networks

“…It depends on the replica dimension n via the scalar spin product and it reduces to c = c in the limit n → 0. In the limit c → ∞ (scaling J as J/c to keep the local fields in the graph h gr i (σ) ≡ j c ij J ij σ j of O(1)) we again recover c = c to leading order as found in [15]. Similarly to the above, one also finds that taking H s → H s + λ i<j c ij J ij produces the average bond strength on the graph.…”

“…Unfortunately, to measure this directly in our system where bonds are mobile would require us to be able to measure correlations over long length scales within the system (in fact scaling like the average loop length ∼ log(N)), which is technically difficult. To try and finesse this problem, in [15,16], the fraction of misaligned spins was calculated, i.e. the fraction of spins that did not point in the direction of their local field.…”

Stability and dynamical properties of material flow systems on random networks

“…The analysis thus allows for a specific characterisation of the effects of such parameters on the stability or otherwise of the system, and on its dynamical behaviour. In subsequent work one can then build on this approach and add more realism by allowing the graph itself evolve in time [50,51].…”