Statistical mechanics of neural networks near saturation

Amit, Daniel J.; Gutfreund, Hanoch; Sompolinsky, Haim

doi:10.1016/0003-4916(87)90092-3

Cited by 812 publications

(501 citation statements)

References 18 publications

Supporting

Mentioning

470

Contrasting

Unclassified

Order By: Relevance

“…Inspection of the equations (2, 3) indicates that we can formally express the weight corresponding to the design process as an additional replica labeled 0 [14][15][16]21]:…”

Section: Free Energy Of the Modelmentioning

confidence: 99%

How accurate must potentials be for successful modeling of protein folding?

Pande

Grosberg

Tanaka

1995

The Journal of Chemical Physics

View full text Add to dashboard Cite

Protein sequences are believed to have been selected to provide the stability of, and reliable renaturation to, an encoded unique spatial fold. In recently proposed theoretical schemes, this selection is modeled as "minimal frustration," or "optimal energy" of the desirable target conformation over all possible sequences, such that the "design" of the sequence is governed by the interactions between monomers. With replica mean field theory, we examine the possibility to reconstruct the renaturation, or freezing transition, of the "designed" heteropolymer given the inevitable errors in the determination of interaction energies, that is, the difference between sets (matrices) of interactions governing chain design and conformations, respectively. We find that the possibility of folding to the designed conformation is controlled by the correlations of the elements of the design and renaturation interaction matrices; unlike random heteropolymers, the ground state of designed heteropolymers is sufficiently stable, such that even a substantial error in the interaction energy should still yield correct renaturation.

show abstract

“…Inspection of the equations (2, 3) indicates that we can formally express the weight corresponding to the design process as an additional replica labeled 0 [14][15][16]21]:…”

Section: Free Energy Of the Modelmentioning

confidence: 99%

How accurate must potentials be for successful modeling of protein folding?

Pande

Grosberg

Tanaka

1995

The Journal of Chemical Physics

View full text Add to dashboard Cite

show abstract

“…The slow plasticity dynamics of synapses are driven by competitive and cooperative interactions consequent on the fast dynamics of firing neurons. The model is analysed within a mean-field approximation, in common with many physics-based approaches to neuroscience, ranging from earlier work [11][12][13][14][15][16][17][18][19][20] to more recent developments [55]. Such a mean-field framework is of course appropriate given the lack of knowledge of microscopic details at the neural or synaptic level.…”

Section: Discussionmentioning

confidence: 99%

“…For example, the study of neural networks [6][7][8], while it greatly simplifies biological structures in order to make them tractable, has still been able to make an impact on the parent field. In particular, neural networks such as the Hopfield model [9,10] have been extensively investigated via methods borrowed from the statistical physics of disordered and complex systems [11][12][13][14]. In these models, memories are stored as patterns of neural activities, which correspond both to low-energy states and to attractors of the stochastic dynamics of the model.…”

Section: Introductionmentioning

confidence: 99%

Slow synaptic dynamics in a network: From exponential to power-law forgetting

Luck

Mehta

2014

Phys. Rev. E

View full text Add to dashboard Cite

We investigate a mean-field model of interacting synapses on a directed neural network. Our interest lies in the slow adaptive dynamics of synapses, which are driven by the fast dynamics of the neurons they connect. Cooperation is modelled from the usual Hebbian perspective, while competition is modelled by an original polarity-driven rule. The emergence of a critical manifold culminating in a tricritical point is crucially dependent on the presence of synaptic competition. This leads to a universal 1/t power-law relaxation of the mean synaptic strength along the critical manifold and an equally universal 1/ √ t relaxation at the tricritical point, to be contrasted with the exponential relaxation that is otherwise generic. In turn, this leads to the natural emergence of long-and short-term memory from different parts of parameter space in a synaptic network, which is the most novel and important result of our present investigations.

show abstract

“…The statistical mechanics of large neural networks with the Hebb rule prescription for the synaptic weights has been studied in detail and is now well-understood [1,2]. In this paper, we shall study the statistical mechanics of neural nets with synaptic weights which are constructed according to the weighted Hebb rule.…”

Section: Introductionmentioning

confidence: 99%

Stochastic neural networks with the weighted Hebb rule

Marzban

Viswanathan

1994

Physics Letters A

View full text Add to dashboard Cite

Neural networks with synaptic weights constructed according to the weighted Hebb rule, a variant of the familiar Hebb rule, are studied in the presence of noise(finite temperature), when the number of stored patterns is finite and in the limit that the number of neurons N → ∞. The fact that different patterns enter the synaptic rule with different weights changes the configuration of the free energy surface. For a general choice of weights not all of the patterns are stored as global minima of the free energy function. However, as for the case of the usual Hebb rule, there exists a temperature range in which only the stored patterns are minima of the free energy. In particular, in the presence of a single extra pattern stored with an appropriate weight in the synaptic rule, the temperature at which the spurious minima of the free energy are eliminated is significantly lower than for a similar network without this extra pattern. The convergence time of the network, together with the overlaps of the equilibria of the network with the stored patterns, can thereby be improved considerably.2

show abstract

Statistical mechanics of neural networks near saturation

Cited by 812 publications

References 18 publications

How accurate must potentials be for successful modeling of protein folding?

How accurate must potentials be for successful modeling of protein folding?

Slow synaptic dynamics in a network: From exponential to power-law forgetting

Stochastic neural networks with the weighted Hebb rule

Contact Info

Product

Resources

About