A discrete gradient method to enhance the numerical behaviour of Hopfield networks

Hernández-Solano, Yadira; Atencia, Miguel; Joya, Gonzalo; Hernández, Francisco

doi:10.1016/j.neucom.2014.10.091

Cited by 13 publications

(12 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The system described by (1) is indeed a dynamic system, confirming the existence of an energy function (or Lyapunov function) which decreases through system trajectories [19]. The energy function of the HNN described in (1) can be defined as follows [18]:…”

Section: A Hnnmentioning

confidence: 75%

See 1 more Smart Citation

Hopfield Neural Network Approach for Supervised Nonlinear Spectral Unmixing

Huang

et al. 2016

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

Nonlinear unmixing, which has attracted considerable interest from researchers and developers, has been successfully applied in many real-world hyperspectral imaging scenarios. Hopfield neural network (HNN) machine learning has already proven successful in solving the linear mixture model; this study utilized an HNN machine learning approach to solve the generalized bilinear model (GBM) optimization problem. Two HNNs were constructed in a successive manner to solve respective seminonnegative matrix factorization problems intended for abundance and nonlinear coefficient estimation. In the proposed HNN-based GBM unmixing method, both HNNs evolve to stable states after a number of iterations to obtain unmixing results related to the states of neurons. In experiments on synthetic data, the proposed method showed more efficient performance in regard to abundance estimation accuracy than other GBM optimization algorithms, especially when given reliable endmember spectra. The proposed method was also applied to real hyperspectral data and still demonstrated notable advantages despite the obvious increase in unmixing difficulty.Index Terms-Generalized bilinear model (GBM), Hopfield neural network (HNN), nonlinear unmixing.

show abstract

Section: A Hnnmentioning

confidence: 75%

“…The key to utilizing a continuous HNN as an optimization method is matching the objective function to the energy function and then constructing the HNN accordingly [19].…”

Section: A Hnnmentioning

confidence: 99%

Hopfield Neural Network Approach for Supervised Nonlinear Spectral Unmixing

Huang

et al. 2016

IEEE Geosci. Remote Sensing Lett.

View full text Add to dashboard Cite

show abstract

“…Celledoni et al [13] extended the Itoh-Abe discrete gradient method to optimisation problems defined on Riemannian manifolds. Hernández-Solano et al [29] combined a discrete gradient method with Hopfield networks in order to preserve a Lyapunov function for optimisation problems.…”

Section: Related Literaturementioning

confidence: 99%

Bregman Itoh–Abe Methods for Sparse Optimisation

2020

View full text Add to dashboard Cite

In this paper we propose optimisation methods for variational regularisation problems based on discretising the inverse scale space flow with discrete gradient methods. Inverse scale space flow generalises gradient flows by incorporating a generalised Bregman distance as the underlying metric. Its discrete-time counterparts, Bregman iterations and linearised Bregman iterations are popular regularisation schemes for inverse problems that incorporate a priori information without loss of contrast. Discrete gradient methods are tools from geometric numerical integration for preserving energy dissipation of dissipative differential systems. The resultant Bregman discrete gradient methods are unconditionally dissipative and achieve rapid convergence rates by exploiting structures of the problem such as sparsity. Building on previous work on discrete gradients for non-smooth, non-convex optimisation, we prove convergence guarantees for these methods in a Clarke subdifferential framework. Numerical results for convex and non-convex examples are presented. Keywords Non-convex optimisation • Non-smooth optimisation • Bregman iteration • Inverse scale space • Geometric numerical integration • Discrete gradient methods Mathematics Subject Classification 49M37 • 49Q15 • 65K10 • 90C26 This article is dedicated to Mila Nikolova whose keen interest and inspiring discussions have encouraged our research on geometric integration for optimisation. MB acknowledges support from the Leverhulme Trust Early Career Fellowship ECF-2016-611 'Learning from mistakes: a supervised feedback-loop for imaging applications'. ESR and CBS acknowledge support from CHiPS (Horizon 2020 RISE project grant) and from the Cantab Capital Institute for the Mathematics of Information. CBS acknowledges support from the Leverhulme Trust project on "Breaking the non-convexity barrier" EPSRC grant Nr EP/M00483X/1, and the EPSRC Centre Nr EP/N014588/1. Moreover, CBS acknowledges support from the RISE project NoMADS and the Alan Turing Institute.

show abstract

“…The original HNN can only deal with the discrete binary pattern recognition by using Hebb's rule [17] and its memory capacity is limited to the network size [18]. However, in recent years lots of works [19][20][21][22][23][24][25][26] have studied the memory capacity and invented different kinds of continuous HNN to deal with the continuous value pattern. We leverage HNN's advantage in warping pattern recognition and the segmentation method in our work, proposing a training-based pattern matching approach, which only needs to be trained on the predefined template pattern.…”

Section: Introductionmentioning

confidence: 99%

An Application of the Associate Hopfield Network for Pattern Matching in Chart Analysis

Mai

Lee

2021

Applied Sciences

View full text Add to dashboard Cite

Chart patterns are significant for financial market behavior analysis. Lots of approaches have been proposed to detect specific patterns in financial time series data, most of them can be categorized as distance-based or training-based. In this paper, we applied a trainable continuous Hopfield Neural Network for financial time series pattern matching. The Perceptually Important Points (PIP) segmentation method is used as the data preprocessing procedure to reduce the fluctuation. We conducted a synthetic data experiment on both high-level noisy data and low-level noisy data. The result shows that our proposed method outperforms the Template Based (TB) and Euclidean Distance (ED) and has an advantage over Dynamic Time Warping (DTW) in terms of the processing time. That indicates the Hopfield network has a potential advantage over other distance-based matching methods.

show abstract

A discrete gradient method to enhance the numerical behaviour of Hopfield networks

Cited by 13 publications

References 32 publications

Hopfield Neural Network Approach for Supervised Nonlinear Spectral Unmixing

Hopfield Neural Network Approach for Supervised Nonlinear Spectral Unmixing

Bregman Itoh–Abe Methods for Sparse Optimisation

An Application of the Associate Hopfield Network for Pattern Matching in Chart Analysis

Contact Info

Product

Resources

About