An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems

Abstract

This paper investigates error-entropy-minimization in adaptive systems training. We prove the equivalence between minimization of error's Renyi entropy of order and minimization of a Csiszar distance measure between the densities of desired and system outputs. A nonparametric estimator for Renyi's entropy is presented, and it is shown that the global minimum of this estimator is the same as the actual entropy. The performance of the error-entropy-minimization criterion is compared with mean-square-error-minimization in the short-term prediction of a chaotic time series and in nonlinear system identification.

Decoding finger movements from ECoG signals using Empirical Mode Decomposition

“…In fact, the minimization of MSE is just taking the secondorder moment of the error distribution into consideration, which is optimal only for Gaussian buted errors. In cases where the error distribution is not Gaussian, it makes sense to study alternate cost functions for adaptation [15]. Here we take a different approach using informationtheoretical concepts, and propose the error entropy C).…”

Signal Processing Techniques Based on Adaptive Radial Basis Function Networks for Chemical Sensor Arrays

“…Parzen window method using a Gaussian kernel [15,16]. The study in [15] demonstrated that the error samples of ITLtrained systems exhibit a more concentrated density function than MSE.…”

Estimating the Information Potential with the Fast Gauss Transform

“…We next consider the system identification of a moving-average model with a 9 th order transfer function given by using the minimization of the error entropy [10]. Although the true advantage of MEE is for nonlinear system identification with nonlinear filters, here the goal is to compare adaptation accuracy and speed so we elected to use a linear plant and a FIR adaptive filter with the same plant order (zero achievable error).…”