Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution

Wu, Shuqiong; Nagahashi, Hiroshi

doi:10.1587/transinf.2015edp7069

Cited by 8 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A logical continuation of this work is quantizing other variants of AdaBoost which depend on domain partitioning hypotheses such as GentleBoost [5], ModestBoost [41], Parameterized AdaBoost [42], and Penalized AdaBoost [43]. Each variant has different generalization abilities, which make them useful in different contexts.…”

Section: Discussionmentioning

confidence: 99%

Quantum Boosting using Domain-Partitioning Hypotheses

Bera¹,

Chatterjee²

2023

Preprint

View full text Add to dashboard Cite

Boosting is an ensemble learning method that converts a weak learner into a strong learner in the PAC learning framework. The AdaBoost algorithm is a well-known classical boosting algorithm for weak learners with binary hypotheses. Recently, Arunachalam and Maity presented the first quantum boosting algorithm by quantizing AdaBoost. Their algorithm, which we refer to as QAdaBoost hereafter, is a quantum adaptation of AdaBoost and only works for the binary hypothesis case. QAdaBoost is quadratically faster than AdaBoost in terms of the VC dimension of the hypothesis class of the weak learner. However, QAdaBoost is polynomially worse in the bias of the weak learner. In this work, we address an open question by Izdebski et al. on the existence of boosting algorithms for quantum weak learners with non-binary hypotheses. We answer this question in the affirmative by developing the QRealBoost algorithm motivated by the classical RealBoost algorithm. The main technical challenge was to provide provable guarantees for convergence, generalization bounds, and quantum speedup, given that quantum subroutines are noisy and probabilistic. We prove that QRealBoost retains the quadratic speedup of QAdaBoost over AdaBoost and further achieves a polynomial speedup over QAdaBoost in terms of both the bias of the learner and the time taken by the learner to learn the target concept class. Finally, we perform empirical evaluations on QRealBoost and report encouraging observations on quantum simulators by benchmarking the convergence performance of QRealBoost against QAdaBoost, AdaBoost, and RealBoost on a subset of the MNIST dataset and Breast Cancer Wisconsin dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

Quantum Boosting using Domain-Partitioning Hypotheses

Bera¹,

Chatterjee²

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Moreover, it improves the thresholding of Marginpruning Boost in Step (2)(e). The parameter in (18) is assigned to [15]. Then we will analyze the generalization abilities of the six variants in the next section.…”

Section: Penalized Adaboostmentioning

confidence: 99%

“…However, its performance is unstable because the accuracy drops occasionally. For the same purpose, Wu and Nagahashi devised Margin-pruning Boost [14] and Penalized AdaBoost [15]. Margin-pruning Boost applies a weight reinitialization approach to reduce the influence from noise-like data, while Penalized AdaBoost improves Margin-pruning Boost by introducing an adaptive weight resetting policy.…”

Section: Introductionmentioning

confidence: 99%

Analysis of Generalization Ability for Different AdaBoost Variants Based on Classification and Regression Trees

Nagahashi

2015

Journal of Electrical and Computer Engineering

Self Cite

View full text Add to dashboard Cite

As a machine learning method, AdaBoost is widely applied to data classification and object detection because of its robustness and efficiency. AdaBoost constructs a global and optimal combination of weak classifiers based on a sample reweighting. It is known that this kind of combination improves the classification performance tremendously. As the popularity of AdaBoost increases, many variants have been proposed to improve the performance of AdaBoost. Then, a lot of comparison and review studies for AdaBoost variants have also been published. Some researchers compared different AdaBoost variants by experiments in their own fields, and others reviewed various AdaBoost variants by basically introducing these algorithms. However, there is a lack of mathematical analysis of the generalization abilities for different AdaBoost variants. In this paper, we analyze the generalization abilities of six AdaBoost variants in terms of classification margins. The six compared variants are Real AdaBoost, Gentle AdaBoost, Modest AdaBoost, Parameterized AdaBoost, Margin-pruning Boost, and Penalized AdaBoost. Finally, we use experiments to verify our analyses.

show abstract

“…The word "additive" doesn't indicate a model fit which is additive in the covariates, but points to the fact that boosting is an additive (in fact, a linear) combination of "simple" (function) estimators. Also, Ratliff et al [9] and Wu et al [10] established related ideas that were mostly acknowledged in the machine learning community. In [11], further views on boosting are given; specifically, the authors first pointed out the relation between boosting and L 1 -penalized estimation.…”

Section: Introductionmentioning

confidence: 99%

A Coherent Performance for Noncoherent Wireless Systems Using AdaBoost Technique

et al. 2019

View full text Add to dashboard Cite

Boosting is a machine learning approach built upon the idea of producing a highly precise prediction rule by combining many relatively weak and imprecise rules. The Adaptive Boosting (AdaBoost) algorithm was the first practical boosting algorithm. It remains one of the most broadly used and studied, with applications in many fields. In this paper, the AdaBoost algorithm is utilized to improve the bit error rate (BER) of different modulation techniques. By feeding the noisy received signal into the AdaBoost algorithm, it is able to recover the transmitted data from the noisy signal. Consequently, it reconstructs the constellation diagram of the modulation technique. This is done by removing the noise that affects and changes the signal space of the data. As a result, AdaBoost shows an improvement in the BER of coherently detected binary phase shift keying (BPSK) and quadrature phase shift keying (QPSK). The AdaBoost is next used to improve the BER of the noncoherent detection of the used modulation techniques. The improvement appears in the form of better results of the noncoherent simulated BER in comparison to that of the theoretical noncoherent BER. Therefore, the AdaBoost algorithm is able to achieve a coherent performance for the noncoherent system. The AdaBoost is simulated for several techniques in additive white Gaussian noise (AWGN) and Rayleigh fading channels so, as to verify the improving effect of the AdaBoost algorithm.

show abstract

Penalized AdaBoost: Improving the Generalization Error of Gentle AdaBoost through a Margin Distribution

Cited by 8 publications

References 23 publications

Quantum Boosting using Domain-Partitioning Hypotheses

Quantum Boosting using Domain-Partitioning Hypotheses

Analysis of Generalization Ability for Different AdaBoost Variants Based on Classification and Regression Trees

A Coherent Performance for Noncoherent Wireless Systems Using AdaBoost Technique

Contact Info

Product

Resources

About