Hidden Hypergraphs, Error-Correcting Codes, and Critical Learning in Hopfield Networks

Abstract

In 1943, McCulloch and Pitts introduced a discrete recurrent neural network as a model for computation in brains. The work inspired breakthroughs such as the first computer design and the theory of finite automata. We focus on learning in Hopfield networks, a special case with symmetric weights and fixed-point attractor dynamics. Specifically, we explore minimum energy flow (MEF) as a scalable convex objective for determining network parameters. We catalog various properties of MEF, such as biological plausibility, and then compare to classical approaches in the theory of learning. Trained Hopfield networks can perform unsupervised clustering and define novel error-correcting coding schemes. They also efficiently find hidden structures (cliques) in graph theory. We extend this known connection from graphs to hypergraphs and discover n-node networks with robust storage of 2Ω(n1−ϵ) memories for any ϵ>0. In the case of graphs, we also determine a critical ratio of training samples at which networks generalize completely.

Photonic Stochastic Emergent Storage for deep classification by scattering-intrinsic patterns

“…As the network becomes more densely populated, the interactions between different memory elements can lead to the emergence of unintended and uncontrolled memory states 2 . To address this limitation, recent research has explored various methods to enhance the capacity of neural networks: dilution 6 – 9 , autapses 10 , 11 , and convex probability flow 12 , 13 .…”

The Effect of Blue Noise on the Optimization Ability of Hopfield Neural Network

“…Hopfield is a feedback neural network model (HNN, Hopfield neural network), which is mainly used to solve various optimization problems [3,4] and also has a wide range of uses in real-world applications, such as image processing [5][6][7], predictive classification [8][9][10], and communication [11,12], and Hopfield neural networks are prone to generating random noise due to their physical properties. Based on this common phenomenon in real-world applications, the paper introduces blue noise into Hopfield neural networks to simulate the noise in real-world applications.…”