A boosting algorithm for regression

Bertoni, Alberto; Campadelli, Paola; Parodi, Marianela

doi:10.1007/bfb0020178

Cited by 26 publications

(21 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…weak learner ) on a weighted training sample. The base learner is expected to return at each iteration t a hypothesis h t from the hypothesis set H that has small weighted training error (see (4)) or large edge (see (18)). These hypotheses are then linearly combined to form the final or composite hypothesis f T as in (1).…”

Section: Leveraging As Stagewise Greedy Optimizationmentioning

confidence: 99%

“…17 A Matlab implementation can be downloaded at http://mlg.anu.edu.au/˜raetsch/software. 18 Here we force the w's to be non-negative, which can be done without loss of generality, if the hypothesis set is closed under negation.…”

Section: Regularization Terms and Sparsenessmentioning

confidence: 99%

“…In this section we present a brief survey of several extensions and generalizations, although many others exist, e.g. [76,158,62,18,31,3,155,15,44,52,82,164,66,38,137,147,16,80,185,100,14].…”

Section: Extensionsmentioning

confidence: 99%

“…Probably the first approach to addressing regression in the context of Boosting appeared in [70], where the problem was addressed by translating the regression task into a classification problem (see also [18]). Most of the Boosting algorithms for binary classification described in this review, are based on a greedy stage-wise minimization of a smooth cost function.…”

Section: Regressionmentioning

confidence: 99%

See 3 more Smart Citations

An Introduction to Boosting and Leveraging

Meir

Rätsch

2003

Advanced Lectures on Machine Learning

281

186

View full text Add to dashboard Cite

Abstract. We provide an introduction to theoretical and practical aspects of Boosting and Ensemble learning, providing a useful reference for researchers in the field of Boosting as well as for those seeking to enter this fascinating area of research. We begin with a short background concerning the necessary learning theoretical foundations of weak learners and their linear combinations. We then point out the useful connection between Boosting and the Theory of Optimization, which facilitates the understanding of Boosting and later on enables us to move on to new Boosting algorithms, applicable to a broad spectrum of problems. In order to increase the relevance of the paper to practitioners, we have added remarks, pseudo code, "tricks of the trade", and algorithmic considerations where appropriate. Finally, we illustrate the usefulness of Boosting algorithms by giving an overview of some existing applications. The main ideas are illustrated on the problem of binary classification, although several extensions are discussed.

show abstract

Section: Leveraging As Stagewise Greedy Optimizationmentioning

confidence: 99%

Section: Regularization Terms and Sparsenessmentioning

confidence: 99%

Section: Extensionsmentioning

confidence: 99%

Section: Regressionmentioning

confidence: 99%

See 2 more Smart Citations

An Introduction to Boosting and Leveraging

Meir

Rätsch

2003

Advanced Lectures on Machine Learning

281

186

View full text Add to dashboard Cite

show abstract

“…Although experimental work shows that algorithms related to AdaBoost.R (H, 1997;Ridgeway, Madigan, & Richardson, 1999;Bertoni, Campadelli, & Parodi, 1997) can be effective, it suffers from two drawbacks.…”

Section: Regressionmentioning

confidence: 99%

Untitled

2002

View full text Add to dashboard Cite

Abstract. In this paper we examine ensemble methods for regression that leverage or "boost" base regressors by iteratively calling them on modified samples. The most successful leveraging algorithm for classification is AdaBoost, an algorithm that requires only modest assumptions on the base learning method for its strong theoretical guarantees. We present several gradient descent leveraging algorithms for regression and prove AdaBoost-style bounds on their sample errors using intuitive assumptions on the base learners. We bound the complexity of the regression functions produced in order to derive PAC-style bounds on their generalization errors. Experiments validate our theoretical results.

show abstract