Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism

Zhou, Xinran; Wang, Chunsheng

doi:10.1016/j.neucom.2015.10.033

Cited by 14 publications

(13 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiment compares the performance of KELM-ML algorithm with the existing multi-label classification learning algorithm: Rank-SVM [9], ML-KNN [8], ECC [6]. KELM-ML is deployed on a distributed cluster built on Hadoop-2.7.3 platform.…”

Section: Resultsmentioning

confidence: 99%

“…Certainly, the parameter selection of KELM-ML algorithm is time-consuming. In terms of ECC, ensemble size is set to be 10, and sample ratio is set to be 67% [6]. The "cross -validation" method is used in the training process.…”

Section: Resultsmentioning

confidence: 99%

“…Regarding positive definiteness of kernel ELM output weight matrix, Cholesky decomposition [3] is adopted to solve hidden layer output matrix inversion problem. Cholesky method is successfully applied to time-varying or nonstationary systems, while incremental updating of the system is implemented [4][5][6]. Inspired by this method, kernel extreme learning machine is applied to multi-label classification problem, which is called KELM-ML.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Multi-label Learning Based on Kernel Extreme Learning Machine

Guo¹,

Huang²,

Chen³

2018

dtcse

View full text Add to dashboard Cite

In recent years, with the increase of data scale, multi-label learning with large scale class labels has turned out to be the research hotspots. Due to the huge solution space, the problem becomes more complex. Therefore, we propose a multi-label algorithm based on kernel learning machine in this paper. Besides, the Cholesky matrix decomposition inverse method is adopted to calculate the network output weight of the kernel extreme learning machine. In particular, in terms of large matrix inverse problem, the large matrix is divided into small matrices for parallel computation through using matrix block method. Compared with several state-of -the-art algorithms on several benchmark data sets, results of the experiments show that the proposed algorithm makes a better performance with large scale class labels.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multi-label Learning Based on Kernel Extreme Learning Machine

Guo¹,

Huang²,

Chen³

2018

dtcse

View full text Add to dashboard Cite

show abstract

“…Zhou et al and Javier [26][27][28] et al applied ELM to remote sensing images. Zhou et al [29] proposed various improvements for ELM to solve the problems in online continuous data applications. Recently, researchers have combined ELM and dimensionality reduction techniques for application.…”

Section: Introductionmentioning

confidence: 99%

Globality‐Locality Preserving Maximum Variance Extreme Learning Machine

et al. 2019

View full text Add to dashboard Cite

An extreme learning machine (ELM) is a useful technique for machine learning; however, the existing extreme learning machine methods cannot exploit the geometric structure information or discriminate information of the data space well. Therefore, we propose a globality-locality preserving maximum variance extreme learning machine (GLELM) based on manifold learning. Based on the characteristics of the traditional ELM method, GLELM introduces the basic principles of linear discriminant analysis (LDA) and local preservation projection (LPP) into ELM, fully taking account of the discriminant information contained in the sample. This method can preserve the global and local manifold structures of data to optimize the projection direction of the classifier. Experiments on several widely used image databases and UCI datasets validate the performance of GLELM. The experimental results show that the proposed model achieves promising results compared to several state-of-the-art ELM algorithms.

show abstract

“…RELM has a better generalization ability by introducing parameters to weigh structural risks and empirical risks [12][13][14][15]. For the single hidden layer RELM model with multiple input and single output, the literature [16] designed the Cholesky factorization method for regularized output weight matrix. In the learning and forgetting process of the sample sequence, the Cholesky factorization factor is calculated recursively by adding and deleting samples one by one, and then the output weights are adjusted, and the network structure is fixed.…”

Section: Introductionmentioning

confidence: 99%

Incremental Multiple Hidden Layers Regularized Extreme Learning Machine Based on Forced Positive‐Definite Cholesky Factorization

Liu

2019

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The theory and implementation of extreme learning machine (ELM) prove that it is a simple, efficient, and accurate machine learning method. Compared with other single hidden layer feedforward neural network algorithms, ELM is characterized by simpler parameter selection rules, faster convergence speed, and less human intervention. The multiple hidden layer regularized extreme learning machine (MRELM) inherits these advantages of ELM and has higher prediction accuracy. In the MRELM model, the number of hidden layers is randomly initiated and fixed, and there is no iterative tuning process. However, the optimal number of hidden layers is the key factor to determine the generalization ability of MRELM. Given this situation, it is obviously unreasonable to determine this number by trial and random initialization. In this paper, an incremental MRELM training algorithm (FC-IMRELM) based on forced positive-definite Cholesky factorization is put forward to solve the network structure design problem of MRELM. First, an MRELM-based prediction model with one hidden layer is constructed, and then a new hidden layer is added to the prediction model in each training step until the generalization performance of the prediction model reaches its peak value. Thus, the optimal network structure of the prediction model is determined. In the training procedure, forced positive-definite Cholesky factorization is used to calculate the output weights of MRELM, which avoids the calculation of the inverse matrix and Moore-Penrose generalized inverse of matrix involved in the training process of hidden layer parameters. Therefore, FC-IMRELM prediction model can effectively reduce the computational cost brought by the process of increasing the number of hidden layers. Experiments on classification and regression problems indicate that the algorithm can be effectively used to determine the optimal network structure of MRELM, and the prediction model training by the algorithm has excellent performance in prediction accuracy and computational cost.

show abstract

Cholesky factorization based online regularized and kernelized extreme learning machines with forgetting mechanism

Cited by 14 publications

References 21 publications

Multi-label Learning Based on Kernel Extreme Learning Machine

Multi-label Learning Based on Kernel Extreme Learning Machine

Globality‐Locality Preserving Maximum Variance Extreme Learning Machine

Incremental Multiple Hidden Layers Regularized Extreme Learning Machine Based on Forced Positive‐Definite Cholesky Factorization

Contact Info

Product

Resources

About