Effective algorithms of the Moore-Penrose inverse matrices for extreme learning machine

Abstract

Extreme learning machine (ELM) is a learning algorithm for single-hidden layer feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs. After the input weights and the hidden layer biases are chosen randomly, ELM can be simply considered a linear system. However, the learning time of ELM is mainly spent on calculating the Moore-Penrose inverse matrices of the hidden layer output matrix. This paper focuses on effective computation of the Moore-Penrose inverse matrices for ELM, several methods are proposed. They are the reduced QR factorization with column Pivoting and Geninv ELM (QRGeninv-ELM), tensor product matrix ELM (TPM-ELM). And we compare QRGeninv-ELM, TPM-ELM with the relational algorithm of Moore-Penrose inverse matrices for ELM, the relational algorithms are: Cholesky factorization of singular matrix ELM (Geninv-ELM), QR factorization and Ginv ELM (QRGinv-ELM), the conjugate Gram-Schmidt process ELM (CGS-ELM). The experimental results and the statistical analysis of the experimental results both demonstrate that QRGeninv-ELM, TPM-ELM and Geninv-ELM are faster than other kinds of ELM and can reach comparable generalization performance.

Calculating the Weighted Moore–Penrose Inverse by a High Order Iteration Scheme

“…Let us consider that A is an m × n matrix whose WSVD is provided by (4). Furthermore, assume that the starting value is given by (11). Thus, the matrix sequence from (15) tends to A † MN .…”

Cognitive concept learning from incomplete information

“…For example, how to apply them in the real world? It includes the semantic explanation of approximate cognitive concepts, the assignment of the parameters k and l, the evaluation of the learnt granular concepts, and how to improve the learning efficiency [17]. Moreover, cognitive logic [26,50] should be incorporated into cognitive concept learning, and uncertainty [29] needs to be considered in incomplete information [29].…”

Applying Cost-Sensitive Extreme Learning Machine and Dissimilarity Integration to Gene Expression Data Classification

“…Tian et al [18, 19] used the Bagging Integrated Model and the modified AdaBoost RT to modify the conventional ELM, respectively. Lu et al [20] proposed several algorithms to reduce the computational cost of the Moore-Penrose inverse matrices for ELM. Zhang et al [21] introduced an incremental ELM which combines the deep feature extracting ability of Deep Learning Networks with the feature mapping ability of the ELM.…”