Backpropagation Learning for Multilayer Feed-Forward Neural Networks Using the Conjugate Gradient Method

Abstract

In many applications, the number of interconnects or weights in a neural network is so large that the learning time for the conventional backpropagation algorithm can become excessively long. Numerical optimization theory offers a rich and robust set of techniques which can be applied to neural networks to improve learning rates. In particular, the conjugate gradient method is easily adapted to the backpropagation learning problem. This paper describes the conjugate gradient method, its application to the backpropagation learning problem and presents results of numerical tests which compare conventional backpropagation, steepest descent and the conjugate gradient methods. For the parity problem, we find that the conjugate gradient method is an order of magnitude faster than conventional backpropagation with momentum.

Modeling electrical properties for various geometries of antidots on a superconducting film

“…Few of its variants include Scaled CG (SCG) and Fletcher-Powell CG (CGF) Johansson et al 1991;Powell 1977). Let us consider set of input vectors r k , which is mutually conjugate with respect to positive definite Hessian matrix H wb , according to condition:…”

Artificial Neural Networks Based Red Palm Weevil (<i>Rynchophorus Ferrugineous, Olivier</i>) Recognition System

“…Feedforward supervised network architecture is mostly used with variant of Backpropogation Algorithm to solve the object recognition problems (Gonzalez and Woods, 2002). The literature review suggested that scaled conjugate gradient Algorithm and Conjugate Gradient with Powell/Beale Restart Method Algorithm performs well in object recognition applications (Johansson et al, 1991;Moller, 1993;Powell, 1977;Beale et al, 2010).…”

Upper Bound on Pattern Storage in Feedforward Networks

“…MLP networks of different sizes were trained using the conjugate gradient algorithm [15], [14], [16] on the data set. A plot of the MSE for different network sizes is shown in Fig.…”