Minimum error entropy criterion‐based randomised autoencoder

Abstract

The extreme learning machine-based autoencoder (ELM-AE) has attracted a lot of attention due to its fast learning speed and promising representation capability. However, the existing ELM-AE algorithms only reconstruct the original input and generally ignore the probability distribution of the data. The minimum error entropy (MEE), as an optimal criterion considering the distribution statistics of the data, is robust in handling non-linear systems and non-Gaussian noises. The MEE is equivalent to the minimisation of the Kullback-Leibaler divergence. Inspired by these advantages, a novel randomised AE is proposed by adopting the MEE criterion as the loss function in the ELM-AE (in short, the MEE-RAE) in this study. Instead of solving the output weight by the Moore-Penrose generalised inverse, the optimal output weight is obtained by the fixed-point iteration method. Further, a quantised MEE (QMEE) is applied to reduce the computational complexity of. Simulations have shown that the QMEE-RAE not only achieves superior generalisation performance but is also more robust to non-Gaussian noises than the ELM-AE.This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited.

Aircraft trajectory filtering method based on Gaussian‐sum and maximum correntropy square‐root cubature Kalman filter

“…Correntropy is a new concept that measures the similarity of two random variables [19, 20, 26]. Consider two random variables X and Y , with a given joint density function $${F}_{X,Y}\left(x,y\right)$$, and the correntropy $$C\left(X,Y\right)$$ is defined as the integral of the error under the MCC, that is, the kernel transformation of the error.…”

“…[19, 26] applied the UKF method, which improves the objective function with the smallest covariance in the Kalman filter and processes the error in the form of a kernel function. This idea is also applied to neural network data processing methods [20], proving that the entropy theory can get more information and more accurate results. Figure 2 shows the MCC kernel function $$\kappa \left(x,y\right)$$ in the space of x and y .…”

“…filter and processes the error in the form of a kernel function. This idea is also applied to neural network data processing methods [20], proving that the entropy theory can get more information and more accurate results. Figure 2 shows the MCC kernel function κ x; y ð Þ in the space of x and y.…”

Multichannel Matrix Randomized Autoencoder

“…Recently, the randomized autoencoder (RAE) especially for the randomized neural network (RNN) [1][2][3][4] based autoencoder (RNN-AE) has attracted much attention due to its advantages of fast learning speed, ease of implementation and less human-intervention [5][6][7][8][9][10][11][12][13][14][15][16]. The RNN-AE can be tracked to [5] that uses random hidden-layer parameters without tuning and only trains the output weight for representation learning.…”