Generalized Guerra’s interpolation schemes for dense associative neural networks

Agliari, Elena; Alemanno, Francesco; Barra, Adriano; Fachechi, Alberto

doi:10.1016/j.neunet.2020.05.009

Cited by 32 publications

(25 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Despite being interesting by itself, this duality could be a promising tool for investigating the principles of information processing in biological phenomena. Indeed, as firstly suggested in [84], Statistical Mechanics turned out to be an effective tool in the description of universal phenomena in biological systems, see also [7,12,16,29,57,63,72,78]. For instance, kinetics of biochemical reactions can be framed in a purely statistical mechanics picture in terms of the Curie-Weiss ferromagnetic model, see [8,11].…”

Section: Introductionmentioning

confidence: 98%

PDE/Statistical Mechanics Duality: Relation Between Guerra’s Interpolated p-Spin Ferromagnets and the Burgers Hierarchy

Fachechi

2021

J Stat Phys

Self Cite

View full text Add to dashboard Cite

We examine the duality relating the equilibrium dynamics of the mean-field p-spin ferromagnets at finite size in the Guerra’s interpolation scheme and the Burgers hierarchy. In particular, we prove that—for fixed p—the expectation value of the order parameter on the first side w.r.t. the generalized partition function satisfies the $$p-1$$ p - 1 -th element in the aforementioned class of nonlinear equations. In the light of this duality, we interpret the phase transitions in the thermodynamic limit of the statistical mechanics model with the development of shock waves in the PDE side. We also obtain the solutions for the p-spin ferromagnets at fixed N, allowing us to easily generate specific solutions of the corresponding equation in the Burgers hierarchy. Finally, we obtain an effective description of the finite N equilibrium dynamics of the $$p=2$$ p = 2 model with some standard tools in PDE side.

show abstract

Section: Introductionmentioning

confidence: 98%

PDE/Statistical Mechanics Duality: Relation Between Guerra’s Interpolated p-Spin Ferromagnets and the Burgers Hierarchy

Fachechi

2021

J Stat Phys

Self Cite

View full text Add to dashboard Cite

show abstract

“…Besides the Mattis magnetizations, that quantify the retrieval of the patterns, we need other parameters to measure glassy behaviors; as standard within the mean-field theory for spin-glasses we define the following so-called 2-replica overlaps (see e.g., [20])…”

Section: The Three-layer Boltzmann Machine and Its Training Algorithmmentioning

confidence: 99%

Learning and Retrieval Operational Modes for Three-Layer Restricted Boltzmann Machines

Agliari

Sebastiani

2021

J Stat Phys

Self Cite

View full text Add to dashboard Cite

We consider a three-layer restricted Boltzmann machine, where the two visible layers (encoding for input and output, respectively) are made of binary neurons while the hidden layer is made of Gaussian neurons, and we show a formal equivalence with a Hopfield model. The machine architecture allows for different learning and operational modes: when all neurons are free to evolve we recover a standard Hopfield model whose size corresponds to the overall size of visible neurons; when input neurons are clamped we recover a Hopfield model, whose size corresponds to the size of the output layer, endowed with an external field as well as additional slow noise. The former stems from the signal provided by the input layer and tends to favour retrieval, the latter can be related to the statistical properties of the training set and tends to impair the retrieval performance of the network. We address this model by rigorous techniques, finding an explicit expression for its free-energy, whence a phase-diagram showing the performance of the system as parameters are tuned.

show abstract

“…where the sum runs over all possible p-tuples and self-interactions are excluded 3 . This kind of model provides an example of dense AMs, which have been intensively studied in the last years (see, e.g., [7,8,20,21]).…”

Section: The P-neuron Hopfield Model With Synaptic Noisementioning

confidence: 99%

Tolerance versus synaptic noise in dense associative memories

Agliari

Marzo

2020

Eur. Phys. J. Plus

Self Cite

View full text Add to dashboard Cite

The retrieval capabilities of associative neural networks are known to be impaired by fast noise, which endows neuron behavior with some degree of stochasticity, and by slow noise, due to interference among stored memories; here, we allow for another source of noise, referred to as “synaptic noise,” which may stem from i. corrupted information provided during learning, ii. shortcomings occurring in the learning stage, or iii. flaws occurring in the storing stage, and which accordingly affects the couplings among neurons. Indeed, we prove that this kind of noise can also yield to a break-down of retrieval and, just like the slow noise, its effect can be softened by relying on density, namely by allowing p-body interactions among neurons.

show abstract

Generalized Guerra’s interpolation schemes for dense associative neural networks

Cited by 32 publications

References 35 publications

PDE/Statistical Mechanics Duality: Relation Between Guerra’s Interpolated p-Spin Ferromagnets and the Burgers Hierarchy

PDE/Statistical Mechanics Duality: Relation Between Guerra’s Interpolated p-Spin Ferromagnets and the Burgers Hierarchy

Learning and Retrieval Operational Modes for Three-Layer Restricted Boltzmann Machines

Tolerance versus synaptic noise in dense associative memories

Contact Info

Product

Resources

About