Numerical Implementation of Gradient Algorithms

Abstract

Abstract. A numerical method for computational implementation of gradient dynamical systems is presented. The method is based upon the development of geometric integration numerical methods, which aim at preserving the dynamical properties of the original ordinary differential equation under discretization. In particular, the proposed method belongs to the class of discrete gradients methods, which substitute the gradient of the continuous equation with a discrete gradient, leading to a map that possesses the same Lyapunov function of the dynamical system, thus preserving the qualitative properties regardless of the step size. In this work, we apply a discrete gradient method to the implementation of Hopfield neural networks. Contrary to most geometric integration methods, the proposed algorithm can be rewritten in explicit form, which considerably improves its performance and stability. Simulation results show that the preservation of the Lyapunov function leads to an improved performance, compared to the conventional discretization.

A discrete gradient method to enhance the numerical behaviour of Hopfield networks

“…Note that Equation (5), which defined the Hopfield network as a single ODE by eliminating the internal variable u, was already in the linear gradient form of Equation (23), as long as we define L(y) as the diagonal matrix…”

“…In Section 5 we considerably expand previous results [23] by presenting some numerical experiments that show the favourable performance of discrete gradient methods, compared to the conventional discretization. Although the theoretical results of previous sections concern the Hopfield network whose self-weights vanish, simulations also include the model with self-weights in order to illustrate the generality of the proposal.…”

Numerical Methods That Preserve a Lyapunov Function for Ordinary Differential Equations

“…Thus the development of discrete gradient methods for dissipative, rather than conservative, systems is limited, and examples of systematic application to real systems are hardly found in the literature, as far as we know. In previous work, we explored the application of discrete gradient methods to a particular system, namely Hopfield neural networks [23,24], which are computational methods used for optimization.…”