A Neural Network with Low Symmetric Connectivity

Abstract

We present the complete, equilibrium solution of a neural-network model in which each neuron is connected to a small fraction of the others by symmetric, Hebb-rule synapses. At first replica symmetry is assumed, but the results are then corrected for full symmetry breaking, which leads to a substantial increase in storage capacity.A breakthrough in the field of neural networks was made by Hopfield [l], who pointed out a mapping between them and spin-glasses. Under this mapping, the average steadystate neural firing patterns correspond to equilibrium states of the corresponding spin system. Spin-glasses themselves are solved using an iterative procedure E21 which is *believed to be exact>> for the SK spin glass [3] (for a review of the technique in this and other disordered problems see [4]). When the method was applied to highly connected neural networks, however, it transpired that only the first, simple steps need be taken to obtain a good result, although a complete solution may be very difficult [5,6].

Untitled

“…For Q = 2 they coincide with those derived via thermodynamical methods [10]. For Q ≥ 3 they are not given in the literature before.…”

“…This means that some of the discrete noise part is neglected. We show that for Q = 2 this procedure leads to the same fixed-point equations as those found from a thermodynamic replica symmetric mean-field theory approach in [10]. For Q > 2, however, these fixed-point equations are new since no replica results are available in the literature.…”

“…In the Q = 2 case the temperature-capacity phase diagram given by a thermodynamic replica-symmetric mean-field theory approach has been presented in [10]. From that work we know that the critical capacity at zero temperature equals 0.634.…”

“…For the special case of Q = 2 the resulting equations (37)-(39) are the same as those derived from a thermodynamic replica-symmetric mean-field theory treatment in [10]. For general Q > 2 such a comparison can not be made because a thermodynamic treatment is not yet available in the literature.…”

Analytic solution of attractor neural networks on scale-free graphs

“…These models were solvable by virtue of the specific nature of their architectures: one either chooses strictly symmetric dilution (so detailed balance and hence equilibrium analysis are preserved, e.g. [9,10,11]), or strictly asymmetric dilution, which ensures that neuron states are statistically independent on finite times [8] (now the local fields are described by Gaussian distributions, leading to simple dynamic order parameter equations). In the early models, the bond statistics were uniform over the entire network, leading to thin tails in its degree distribution, whereas the connectivity of a real neuron is known to vary strongly within the brain [12].…”