Boosting for Control of Dynamical Systems

Agarwal, Naman; Brukhim, Nataly; Hazan, Elad; Lu, Zhou

doi:10.48550/arxiv.1906.08720

Cited by 2 publications

(2 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[3,4,24] and referenced work therein. The online learning approach also gave rise to the first boosting methods in this context [2], and demonstrates the potential impact of boosting in the online setting. Thus, the current work aims at continuing the development of the boosting methodology in online machine learning, starting from the basic setting of expert advice.…”

Section: Introductionmentioning

confidence: 84%

Online Agnostic Boosting via Regret Minimization

Brukhim,

Chen,

Hazan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Boosting is a widely used machine learning approach based on the idea of aggregating weak learning rules. While in statistical learning numerous boosting methods exist both in the realizable and agnostic settings, in online learning they exist only in the realizable case. In this work we provide the first agnostic online boosting algorithm; that is, given a weak learner with only marginally-better-than-trivial regret guarantees, our algorithm boosts it to a strong learner with sublinear regret.Our algorithm is based on an abstract (and simple) reduction to online convex optimization, which efficiently converts an arbitrary online convex optimizer to an online booster. Moreover, this reduction extends to the statistical as well as the online realizable settings, thus unifying the 4 cases of statistical/online and agnostic/realizable boosting.

show abstract

Section: Introductionmentioning

confidence: 84%

Online Agnostic Boosting via Regret Minimization

Brukhim,

Chen,

Hazan

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A famous example which fits this paradigm is the Viola Jones algorithm [Viola and Jones, 2001], which uses simple rectangular-based prediction rules for the task of object detection. In the real-valued online learning setting this point of view is also adopted [Beygelzimer et al, 2015, Agarwal et al, 2019.…”

Section: Introductionmentioning

confidence: 99%

Boosting Simple Learners

Alon¹,

Gonen²,

Hazan³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

We consider boosting algorithms under the restriction that the weak learners come from a class of bounded VC-dimension. In this setting, we focus on two main questions:(i) Oracle Complexity: we show that the restriction on the complexity of the weak learner significantly improves the number of calls to the weak learner. We describe a boosting procedure which makes only Õ(1/γ) calls to the weak learner, where γ denotes the weak learner's advantage. This circumvents a lower bound of Ω(1/γ 2 ) due to Freund and Schapire ('95, '12) for the general case. Unlike previous boosting algorithms which aggregate the weak hypotheses by majority votes, our method use more complex aggregation rules, and we show this to be necessary.(ii) Expressivity: we consider the question of what can be learned by boosting weak hypotheses of bounded VC-dimension? Towards this end we identify a combinatorial-geometric parameter called the γ-VC dimension which quantifies the expressivity of a class of weak hypotheses when used as part of a boosting procedure. We explore the limits of the γ-VC dimension and compute it for well-studied classes such as halfspaces and decision stumps. Along the way, we establish and exploit connections with Discrepancy theory.

show abstract

Boosting for Control of Dynamical Systems

Cited by 2 publications

References 8 publications

Online Agnostic Boosting via Regret Minimization

Online Agnostic Boosting via Regret Minimization

Boosting Simple Learners

Contact Info

Product

Resources

About