A Novel AdaBoost Framework With Robust Threshold and Structural Optimization

Abstract

The AdaBoost algorithm is a popular ensemble method that combines several weak learners to boost generalization performance. However, conventional AdaBoost.RT algorithms suffer from the limitation that the threshold value must be manually specified rather than chosen through a self-adaptive mechanism, which cannot guarantee a result in an optimal model for general cases. In this paper, we present a generic AdaBoost framework with robust threshold mechanism and structural optimization on regression problems. The error statistics of each weak learner on one given problem dataset is utilized to automate the choice of the optimal cut-off threshold value. In addition, a special single-layer neural network is employed to provide a second opportunity to further adjust the structure and strength the adaption capability of the AdaBoost regression model. Moreover, to consolidate the theoretical foundation of AdaBoost algorithms, we are the first to conduct a rigorous and comprehensive theoretical analysis on the proposed approach. We prove that the general bound on the empirical error with a fraction of training examples is always within a limited soft margin, which indicates that our novel algorithm can avoid over-fitting. We further analyze the bounds on the generalization error directly under probably approximately correct learning. The extensive experimental verifications on the UCI benchmarks have demonstrated that the performance of the proposed method is superior to other state-of-the-art ensemble and single learning algorithms. Furthermore, a real-world indoor positioning application has also revealed that the proposed method has higher positioning accuracy and faster speed.

An ELM-Embedded Deep Learning Based Intelligent Recognition System for Computer Numeric Control Machine Tools

“…Although ELM is a relatively recent intelligent technique, the theories and related applications have been investigated extensively, such as kernel ELM, incremental ELM (I-ELM), Bayesian ELM, adaboost ELM [30], multi-layer ELM-LRF [31], hybrid ELMs [32], etc. Particularly, ELM auto-encoder (ELM-AE) [33] is proposed to introduce unsupervised learning for ELM, by projecting the input data into a different dimensional space.…”

Ensemble Learning with Stochastic Configuration Network for Noisy Optical Fiber Vibration Signal Recognition

“…This shows that the SCN after training is less robust in this case. Combining models with ensemble learning methods is a common choice in solving the problem of low robustness [26,27]. In the field of ensemble learning methods, Boosting and Bagging are two representative types of methods.…”

An SVM-Based AdaBoost Cascade Classifier for Sonar Image

“…By directly minimizing the computational cost of the cascade, it searched for the best partition point of each stage and provided theoretical support to ensure the existence of a unique optimal solution. P Zhang [25] introduced a single-layer neural network to optimize the threshold based on the adaptive ELM with Sshaped activation function, and also employed a grid search strategy to select the regularization parameter C from a wide range. W M [26] used AdaBoost method to find samples corresponding with larger weights, and removed them as possible outliers, and then retrained and redesigned the classifier model.…”