QR factorization based Incremental Extreme Learning Machine with growth of hidden nodes

The proposal of this study is a fast version of the conventional extreme learning machine (ELM), called pseudoinverse matrix decomposition based incremental ELM (PDI-ELM). One of the main problems in ELM is to determine the number of hidden nodes. In this study, the number of hidden nodes is automatically determined. The proposed model is an incremental version of ELM which adds neurons with the goal of minimization the error of the ELM network. To speed up the model the information of pseudoinverse from previous step is taken into account in the current iteration. To show the ability of the PDI-ELM, it is applied to few benchmark classification datasets in the University of California Irvine (UCI) repository. Compared to ELM learner and two other versions of incremental ELM, the proposed PDI-ELM is faster.

show abstract

“…The details regarding how to find computational complexities for three compared models can be found at [13].…”

Section: Experiments Resultsmentioning

confidence: 99%

“…The latest version of I-ELM based methods, called QRI-ELM [13] decomposes the pseudoinverse matrix of the hidden output layer based on QR factorization. Indeed, H=Q.R wherein Q is an orthogonal matrix and R is an upper triangular matrix.…”

Section: Incremental Based Extreme Learning Machinesmentioning

confidence: 99%

Pseudoinverse Matrix Decomposition Based Incremental Extreme Learning Machine with Growth of Hidden Nodes

Kassani

Kim

2016

IJFIS

View full text Add to dashboard Cite

show abstract

“…Ye and Qin [6] use QR factorization to determine the number of hidden nodes in generalized single-hidden-layer feedforward networks. In [7,8], parallel QR factorization algorithms are used in shared memory, synchronous message passing, and asynchronous message passing.…”

Section: Some Properties and Applications Of Matrix Factorizationsmentioning

confidence: 99%

Some properties of various types of matrix factorization

Tan

2021

ITM Web Conf.

View full text Add to dashboard Cite

Matrix factorizations or matrix decompositions are methods that represent a matrix as a product of two or more matrices. There are various types of matrix factorizations such as LU factorization, Cholesky factorization, singular value decomposition etc. Matrix factorization is widely used in pattern recognition, image denoising, data clustering etc. Motivated by these applications, some properties and applications of various types of matrix factorizations are studied. One of the purposes of matrix factorization is to ease the computation. Thus, comparisons in term of computation time of various matrix factorizations in different areas are carried out.

show abstract

“…This approach achieved good speed in the algorithm execution. The latest variant of I-ELM is QRI-ELM [15], which uses QR factorization to decompose pseudoinverse matrix of hidden output layer such as H † = R -1 Q T , where H is the hidden layer output matrix, Q and R are orthogonal and an upper triangular matrix of H respectively. In this way, this approach simplifies the computation of pseudoinverse of hidden layer output matrix (H).…”

Section: Introductionmentioning

confidence: 99%

Rank Based Pseudoinverse Computation in Extreme Learning Machine for large Datasets

Ragala¹,

Kumar²

2019

IJITEE

View full text Add to dashboard Cite

Extreme Learning Machine (ELM) is an efficient and effective least-square-based learning algorithm for classification, regression problems based on single hidden layer feed-forward neural network (SLFN). It has been shown in the literature that it has faster convergence and good generalization ability for moderate datasets. But, there is great deal of challenge involved in computing the pseudoinverse when there are large numbers of hidden nodes or for large number of instances to train complex pattern recognition problems. To address this problem, a few approaches such as EM-ELM, DF-ELM have been proposed in the literature. In this paper, a new rank-based matrix decomposition of the hidden layer matrix is introduced to have the optimal training time and reduce the computational complexity for a large number of hidden nodes in the hidden layer. The results show that it has constant training time which is closer towards the minimal training time and very far from worst-case training time of the DF-ELM algorithm that has been shown efficient in the recent literature.

show abstract

QR factorization based Incremental Extreme Learning Machine with growth of hidden nodes

Cited by 15 publications

References 20 publications

Pseudoinverse Matrix Decomposition Based Incremental Extreme Learning Machine with Growth of Hidden Nodes

Pseudoinverse Matrix Decomposition Based Incremental Extreme Learning Machine with Growth of Hidden Nodes

Some properties of various types of matrix factorization

Rank Based Pseudoinverse Computation in Extreme Learning Machine for large Datasets

Contact Info

Product

Resources

About