A recursive algorithm based on the extended Kalman filter for the training of feedforward neural models

Abstract

To cite this version:Isabelle Rivals, Léon Personnaz. A recursive algorithm based on the extended Kalman filter for the training of feedforward neural models. Neurocomputing, Elsevier, 1998, 20 (1-3), pp.279-294. <hal-00797391> In Neurocomputing (1-3): 279-294 (1998 AbstractThe Extended Kalman Filter (EKF) is a well known tool for the recursive parameter estimation of static and dynamic nonlinear models. In particular, the EKF has been applied to the estimation of the weights of feedforward and recurrent neural network models, i.e. to their training, and shown to be more efficient than recursive and non recursive first-order training algorithms; nevertheless, these first applications to the training of neural networks did not fully exploit the potentials of the EKF.In this paper, we analyze the specific influence of the EKF parameters for modeling problems, and propose a variant of this algorithm for the training of feedforward neural models which proves to be very efficient as compared to non recursive second-order algorithms. We test the proposed EKF algorithm on several static and dynamic modeling problems, some of them being benchmark problems, and which bring out the properties of the proposed algorithm..

Estimating tree bole volume using artificial neural network models for four species in Turkey

“…A Kalman filter attempts to estimate the state of a system that can be modeled as a linear system driven by additive white Gaussian noise and where the measurements available are linear combinations of the system states corrupted by additive white Gaussian noise (Li et al, 2002). For neural network training the weights of the networks are the states process of state equation the Kalman filter attempts to estimate and the desired output of the network is the measurement (measurement equation) used by the Kalman filter as shown in the equations below (Rivals and Personnaz, 1998):…”

Performance evaluation of artificial neural networks in estimating reference evapotranspiration with minimal meteorological data

“…This learning rule is acceptable for forecasting the type of problems where the number of inputs is not too large (Grewal and Andrews, 2001). A Kalman filter (Brown and Hwang, 1992;Rivals and Personnaz, 1998;Demuth and Beale, 2001;Grewal and Andrews, 2001;Li et al, 2002) attempts to estimate the state of a system that can be modeled as a linear system driven by additive white Gaussian noise and where the available measurements are linear combinations of the system states that are corrupted by additive white Gaussian noise (Li et al, 2002). A detailed description of the Cascade Correlation architecture where Kalman's learning rule is embedded can be found in Diamantopoulou (2010).…”

“…This value is used to initialize the diagonal value of the error covariance matrix for the Kalman estimates of the weights. Additionally, the work of Rivals and Personnaz (1998) analyzes the cases of the different initial values for the Kalman Filter training algorithm and provides advice on how to choose the initial values of the system error covariance and process noise error in the Kalman recursion.…”

Robust GMDH-type neural network with unscented Kalman filter for non-linear systems

“…Furthermore, it reduces the dimensions of the state vector. Rivals and Personnaz (1998) investigated various conditions of the initial weights for the EKF training algorithm. Van der Merwe (2004) used the unscented Kalman filter (UKF) to train neural networks and showed that UKF is much more stable and has a faster convergent rate than that of the EKF algorithm (Zhao et al, 2012).…”