Analysis of Generalization Ability for Different AdaBoost Variants Based on Classification and Regression Trees

Abstract

As a machine learning method, AdaBoost is widely applied to data classification and object detection because of its robustness and efficiency. AdaBoost constructs a global and optimal combination of weak classifiers based on a sample reweighting. It is known that this kind of combination improves the classification performance tremendously. As the popularity of AdaBoost increases, many variants have been proposed to improve the performance of AdaBoost. Then, a lot of comparison and review studies for AdaBoost variants have also been published. Some researchers compared different AdaBoost variants by experiments in their own fields, and others reviewed various AdaBoost variants by basically introducing these algorithms. However, there is a lack of mathematical analysis of the generalization abilities for different AdaBoost variants. In this paper, we analyze the generalization abilities of six AdaBoost variants in terms of classification margins. The six compared variants are Real AdaBoost, Gentle AdaBoost, Modest AdaBoost, Parameterized AdaBoost, Margin-pruning Boost, and Penalized AdaBoost. Finally, we use experiments to verify our analyses.

Context‐based ensemble classification for the detection of architectural distortion in a digitised mammogram

“…As mentioned in Section 1, we use an ensemble of classifiers to label an RoI as an AD or a normal site. There are two popular ensemble classifiers, AdaBoost [8,20,21] and Random Forest [9], both of which build a classifier with multiple weak learners. The reader may note that weak learners in AdaBoost are trained on feature vectors that are assigned unequal weights based on their difficulty level in classification.…”

“…An attempt of the detection of AD that looks similar to the example on multiple radiologists is weighing the decisions of multiple classifiers (in place of radiologists) in coming to a final decision regarding a particular RoI. Results on different pattern recognition problems show that an ensemble performs better [8] than a single classifier in certain problems.…”

“…The reader may note that in the machine learning literature there are two popular ensemble classifiers like Adaboost [8] and Random Forest [9]. In this manuscript, a method has been proposed that incorporates the observation of Fig.…”

Extending the Classifier Algorithms in Machine Learning to Improve the Performance in Spoken Language Understanding Systems Under Deficient Training Data

“…In literature AdaBoost are used for detecting objects [26] ,detecting faults in gearbox [27] and detecting hands [28] In another paper, AdaBoost is used in mobile intelligent terminals for security level classification [29] Various modified AdaBoost algorithms are were analyzed for classification margin [30] Further, in the same paper, generalized error for eight variants of AdaBoost was reported. Personalized AdaBoost which is a variant of AdaBoost as reported in the paper has the least generalization error…”

Real Time Weed Detection using a Boosted Cascade of Simple Features

“…α is a weighting factor for each weak classifier g, S being the overall classification of the function threshold G. Different variations of the doping algorithm were developed : Discrete AdaBoost [8] Real AdaBoost [18] Gentle AdaBoost, etc. a recent performance of these alternatives analysis is presented in [19]. We use here the first defined by the following pseudo algorithm: For each descriptor (j), train a classifier (G i ).…”