Replica symmetry breaking in neural networks: a few steps toward rigorous results

Abstract

In this paper we adapt the broken replica interpolation technique (developed by Francesco Guerra to deal with the Sherrington-Kirkpatrick model, namely a pairwise mean-field spin-glass whose couplings are i.i.d. standard Gaussian variables) in order to work also with the Hopfield model (i.e. a pairwise meanfield neural-network whose couplings are drawn according to Hebb's learning rule): this is accomplished by grafting Guerra's telescopic averages on the transport equation technique, recently developed by some of the authors. As an overture, we apply the technique to solve the Sherrington-Kirkpatrick model with i.i.d. Gaussian couplings centered at J 0 and with finite variance J; the mean J 0 provides a ferromagnetic contribution to be detected in a noisy environment tuned by J, hence making this model a natural test-case to be investigated before addressing the Hopfield model. For both the models, an explicit expression of their quenched free energy in terms of their natural order parameters is obtained at the Kth step (K arbitrary, but finite) of replica-symmetry-breaking. In particular, for the Hopfield model, by assuming that the overlaps respect Parisi's decomposition (following the ziqqurat ansatz) and that the Mattis magnetization is self-averaging, we recover previous results obtained via replica-trick by Amit, Crisanti and Gutfreund (1RSB) and by Steffan and Kühn (2RSB).

PDE/Statistical Mechanics Duality: Relation Between Guerra’s Interpolated p-Spin Ferromagnets and the Burgers Hierarchy

“…Also, the application of the whole technology to equilibrium dynamics of neural networks would be highly desirable. To conclude, we strongly believe that PDEstatistical mechanics duality can be, at least in the thermodynamic limit, a very useful tool to investigate properties at the equilibrium of spin-glass models for each interaction order p [33,36,37,45,69,76,77,87]. We leave this discussion open for future works.…”

Rigorous approaches for spin glass and Gaussian spin glass with P-wise interactions

“…Now the aim is to show that, using a different strategy, such as the transport equation technique, already introduced in [1] for both assumption in classic SK model with ferromagnetic contribution, we are able to retrieve the same expression of quenched statistical pressure and self-consistency equations found via Guerra's interpolating technique.…”

“…Definition 12. Given the interpolating parameters r = (x (1) , x (2) , w), t and the i.i.d. auxiliary fields {z…”

“…Proposition 7. At finite size and under 1RSB assumption applying the Fundamental Theorem of Calculus and using the suitable values of A (1) , A (2) , B, we find the quenched pressure for the Gaussian P-spin glass model as…”

“…In this paper we focus on mathematical methods on spin glasses, considered a real challenge for mathematicians [33]. In particular, in this paper we want to generalize some computations for Sherrington Kirkpatrick and Gaussian models with two-spin interactions in a polynomial interactions of degree P > 2, with two different rigorous mathematical approaches, namely Guerra's interpolating technique, introduced in [23,24], and transport equation, introduced in Replica Symmetric assumption in [5] and in Replica Symmetry Breaking [1] for the description of Hopfield model and, afterwards, for deep Hopfield networks [2]. Thus we prove that both methods are consistent and mathematically valid for these models.…”