Weighted patterns as a tool for improving the Hopfield model

Abstract

We generalize the standard Hopfield model to the case when a weight is assigned to each input pattern. The weight can be interpreted as the frequency of the pattern occurrence at the input of the network. In the framework of the statistical physics approach we obtain the saddle-point equation allowing us to examine the memory of the network. In the case of unequal weights our model does not lead to the catastrophic destruction of the memory due to its overfilling (that is typical for the standard Hopfield model). The real memory consists only of the patterns with weights exceeding a critical value that is determined by the weights distribution. We obtain the algorithm allowing us to find this critical value for an arbitrary distribution of the weights, and analyze in detail some particular weights distributions. It is shown that the memory decreases as compared to the case of the standard Hopfield model. However, in our model the network can learn online without the catastrophic destruction of the memory.

Approximation of the n-vicinity method

“…Статфизические подходы широко используются для описания нейронных сетей большого размера [1][2][3][4]. В частности, наиболее продвинутые нейросетевые алгоритмы глубокого обучения и обработки изображений [5,6] основаны на оптимизации свободной энергии системы спинов-нейронов, т. е. на оптимизации распределения энергий.…”

Generalized approach to description of energy distribution of spin system

“…For that, we express the matrix T in the form Taking into account Eqs. (17)(18)(19) we present the quasienergy in the configuration 0 S in the form…”

Dependence of Critical Parameters of 2D Ising Model on Lattice Size

“… (23). states that at different instants of time the magnetization obtained in the framework of simulations takes on any value between 0…”

“…Instead we see that the heat capacity in the critical point increases logarithmically, that is N . However, it is difficult to realize this transition since if we increase the size of the lattice even up to the Avogadro number dependence of the heat capacity on N means violation of the additivity concept for a classical system that is clearly seen even when23 : if we double the size of the system the value of c C is up by 1.3%. Of course, we examined the system with the free boundary conditions and such systems are non-additive by definition.…”