The memory degradation based online sequential extreme learning machine

Zou, Quanyi; Wang, Xiaojun; Zhou, Changjun; Zhang, Qiang

doi:10.1016/j.neucom.2017.11.030

Cited by 14 publications

(3 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extreme learning machine [1,2], as a remarkable single hidden layer feed-forward neural networks (SLFNs) [3] training method, has been widely studied and applied in many fields such as efficient modeling [4], fashion retailing forecasting [5], fingerprint matching [6], metagenomic taxonomic classification [7], online sequential learning [8], and feature selection [9]. The weights of the input layer and hidden layer offsets are randomly generated.…”

Section: Introductionmentioning

confidence: 99%

Robust Fisher-Regularized Twin Extreme Learning Machine with Capped L1-Norm for Classification

Xue

Cai

2023

Axioms

View full text Add to dashboard Cite

Twin extreme learning machine (TELM) is a classical and high-efficiency classifier. However, it neglects the statistical knowledge hidden inside the data. In this paper, in order to make full use of statistical information from sample data, we first come up with a Fisher-regularized twin extreme learning machine (FTELM) by applying Fisher regularization into TELM learning framework. This strategy not only inherits the advantages of TELM, but also minimizes the within-class divergence of samples. Further, in an effort to further boost the anti-noise ability of FTELM method, we propose a new capped L1-norm FTELM (CL1-FTELM) by introducing capped L1-norm in FTELM to dwindle the influence of abnormal points, and CL1-FTELM improves the robust performance of our FTELM. Then, for the proposed FTELM method, we utilize an efficient successive overrelaxation algorithm to solve the corresponding optimization problem. For the proposed CL1-FTELM, an iterative method is designed to solve the corresponding optimization based on re-weighted technique. Meanwhile, the convergence and local optimality of CL1-FTELM are proved theoretically. Finally, numerical experiments on manual and UCI datasets show that the proposed methods achieve better classification effects than the state-of-the-art methods in most cases, which demonstrates the effectiveness and stability of the proposed methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Robust Fisher-Regularized Twin Extreme Learning Machine with Capped L1-Norm for Classification

Xue

Cai

2023

Axioms

View full text Add to dashboard Cite

show abstract

“…Applying invalid samples, online models always export inaccuracy results. To handle this issue, the OS-ELM with forgetting mechanism (FOS-ELM) was proposed by Zhao et al [11], where the forgetting mechanism can discard obsolescence samples and enhance the accuracy of the predictive model. Zou et al [12] proposed the memory degradation based OS-ELM (MDOS-ELM) which adjusts the weights of the old and new samples by a self-adaptive memory factor, and discards invalid samples.…”

Section: Introductionmentioning

confidence: 99%

A Novel Robust Online Extreme Learning Machine for the Non-Gaussian Noise

2023

View full text Add to dashboard Cite

Samples collected from most industrial processes have two challenges: one is contaminated by the non-Gaussian noise, and the other is gradually obsolesced. This feature can obviously reduce the accuracy and generalization of models. To handle these challenges, a novel method, named the robust online extreme learning machine (RO-ELM), is proposed in this paper, in which the least mean -power criterion is employed as the cost function which is to boost the robustness of the ELM, and the forgetting mechanism is introduced to discard the obsolescence samples. To investigate the performance of the RO-ELM, experiments on artificial and real-world datasets with the non-Gaussian noise are performed, and the datasets are from regression or classification problems. Results show that the RO-ELM is more robust than the ELM, the online sequential ELM (OS-ELM) and the OS-ELM with forgetting mechanism (FOS-ELM). The accuracy and generalization of the RO-ELM models are better than those of other models for online learning.

show abstract

“…With the continuous arrival of new data and the repeated training on the past data, the ELM's learning speed gradually declines and disappears, and even worse its learning ability collapses altogether. To solve the above problem, the Online Sequential Extreme Learning Machine (OS-ELM) has been proposed [13][14][15], an online ELM variant. e OS-ELM only trains on newly arriving data and then combines the existing prediction model and the training results to update a new prediction model.…”

Section: Introductionmentioning

confidence: 99%

Local Gravitation Clustering-Based Semisupervised Online Sequential Extreme Learning Machine

Wang

Zou

et al. 2022

Security and Communication Networks

View full text Add to dashboard Cite

Due to the limited number of labeled samples, semisupervised learning often leads to a considerable empirical distribution mismatch between labeled samples and unlabeled samples. To this end, this paper proposes a novel semisupervised algorithm named Local Gravitation-based Semisupervised Online Sequential Extreme Learning Machine (LGS-OSELM), learning to unlabeled samples follows from easy to difficult. Each sample is formulated as an object with mass and associated with local gravitation generated from its neighbors. The similarity between samples is measurable by the local gravitation measures (centrality CE and coordination CO). First, the LGS-OSELM uses the labeled samples to learn the initialization model by implementing ELM. Second, the unlabeled samples with a high confidence level that is easy to learn are labeled with the pseudo label. Then, these samples are utilized to iterate the neural network by implementing OS-ELM. The proposed approach ultimately realizes effective learning of all samples through successive learning unlabeled samples and iterating neural networks. We implement experiments on several standard benchmark data sets to verify the performance of the proposed LGS-OSELM, which demonstrates that our proposed approach outperforms state-of-the-art methods in terms of accuracy.

show abstract

The memory degradation based online sequential extreme learning machine

Cited by 14 publications

References 41 publications

Robust Fisher-Regularized Twin Extreme Learning Machine with Capped L1-Norm for Classification

Robust Fisher-Regularized Twin Extreme Learning Machine with Capped L1-Norm for Classification

A Novel Robust Online Extreme Learning Machine for the Non-Gaussian Noise

Local Gravitation Clustering-Based Semisupervised Online Sequential Extreme Learning Machine

Contact Info

Product

Resources

About