Pareto Optimal Model Selection in Linear Bandits

Zhu, Yinglun; Nowak, Robert

doi:10.48550/arxiv.2102.06593

Cited by 4 publications

(6 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Foster et al [2019] assume the contexts are also drawn from an unknown distribution x ∼ D and propose an algorithm which does not incur more than Õ( 1 γ 3 (i * T ) 2/3 (M d i * ) 1/3 ), where γ 3 is the smallest eigenvalue of the covariance matrix of feature embeddings Σ = E x∼D 1 M a∈A φ M (x, a)φ M (x, a) ⊤ . Pacchiano et al [2020b] propose a different approach based on the corralling algorithm of Agarwal et al [2017] Zhu and Nowak [2021] show that it is impossible to achieve the desired regret guarantees of √ d m * T without additional assumptions by showing a result similar to the one of Lattimore [2015]. The work of Lattimore [2015] states that in the stochastic multi-armed bandit problem it is impossible to achieve √ T regret to a fixed arm, without suffering at least K √ T regret to a different arm.…”

Section: Related Workmentioning

confidence: 96%

The Pareto Frontier of model selection for general Contextual Bandits

Marinov¹,

Zimmert²

2021

Preprint

View full text Add to dashboard Cite

Recent progress in model selection raises the question of the fundamental limits of these techniques. Under specific scrutiny has been model selection for general contextual bandits with nested policy classes, resulting in a COLT2020 open problem. It asks whether it is possible to obtain simultaneously the optimal single algorithm guarantees over all policies in a nested sequence of policy classes, or if otherwise this is possible for a trade-off α ∈ [ 1 2 , 1) between complexity term and time: ln(|Π m |) 1−α T α . We give a disappointing answer to this question. Even in the purely stochastic regime, the desired results are unobtainable. We present a Pareto frontier of up to logarithmic factors matching upper and lower bounds, thereby proving that an increase in the complexity term ln(|Π m |) independent of T is unavoidable for general policy classes. As a side result, we also resolve a COLT2016 open problem concerning second-order bounds in full-information games.

show abstract

Section: Related Workmentioning

confidence: 96%

The Pareto Frontier of model selection for general Contextual Bandits

Marinov¹,

Zimmert²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For example, [27] shows that, in K-armed bandit problems, one cannot simultaneously achieve O( √ T ) regret to one specific arm, while guaranteeing O( √ KT ) regret to the rest. Such results have been extended to both linear and Lipschitz non-contextual bandits [41,30,23] as well as Lipschitz contextual bandits under margin assumptions [20], and they establish that model selection is not possible in these settings. However, these ideas have not been extended to standard contextual bandit settings, which is our focus.…”

Section: Related Workmentioning

confidence: 99%

Universal and data-adaptive algorithms for model selection in linear contextual bandits

Muthukumar¹,

Krishnamurthy²

2021

Preprint

View full text Add to dashboard Cite

Model selection in contextual bandits is an important complementary problem to regret minimization with respect to a fixed model class. We consider the simplest non-trivial instance of model-selection: distinguishing a simple multi-armed bandit problem from a linear contextual bandit problem. Even in this instance, current state-of-the-art methods explore in a suboptimal manner and require strong "feature-diversity" conditions. In this paper, we introduce new algorithms that a) explore in a dataadaptive manner, and b) provide model selection guarantees of the form O(d α T 1−α ) with no feature diversity conditions whatsoever, where d denotes the dimension of the linear model and T denotes the total number of rounds. The first algorithm enjoys a "best-of-both-worlds" property, recovering two prior results that hold under distinct distributional assumptions, simultaneously. The second removes distributional assumptions altogether, expanding the scope for tractable model selection. Our approach extends to model selection among nested linear contextual bandits under some additional assumptions.

show abstract

“…This approach, however, is directly ruled out by Proposition 2 since such doubling trick could end up with solving a problem with a dimension d ≤ 2d yet ρ d ρ d . Although doubling trick over dimensions is commonly used to provide worstcase guarantees in the regret minimization settings [13,21], we emphasize here that matching the instance-dependent complexity measure is a common goal in the pure exploration setting [9,6,16], and thus new techniques need to be developed. Proposition 2 also implies that trying to infer the value of ρ d from ι d can be quite misleading.…”

Section: Failure Of Standard Approachesmentioning

confidence: 99%

“…In fact, crossvalidation [18,17], a practical method for model selection, appears in almost all successful deploy-ments of machine learning models. The model selection problem was recently introduced to the bandit regret minimization setting by [7], and further analyzed by [13,21]. [21] prove that only Pareto optimality can be achieved for regret minimization, which is even weaker than minimax optimality.…”

Section: Introductionmentioning

confidence: 99%

“…The model selection problem was recently introduced to the bandit regret minimization setting by [7], and further analyzed by [13,21]. [21] prove that only Pareto optimality can be achieved for regret minimization, which is even weaker than minimax optimality. We introduce the model selection problem in the pure exploration setting and, surprisingly, show that it is possible to design algorithms with near optimal instance-dependent complexity for both fixed confidence and fixed budget settings.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Near Instance Optimal Model Selection for Pure Exploration Linear Bandits

Zhu¹,

Katz-Samuels²,

Nowak³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The model selection problem in the pure exploration linear bandit setting is introduced and studied in both the fixed confidence and fixed budget settings. The model selection problem considers a nested sequence of hypothesis classes of increasing complexities. Our goal is to automatically adapt to the instance-dependent complexity measure of the smallest hypothesis class containing the true model, rather than suffering from the complexity measure related to the largest hypothesis class. We provide evidence showing that a standard doubling trick over dimension fails to achieve the optimal instance-dependent sample complexity. Our algorithms define a new optimization problem based on experimental design that leverages the geometry of the action set to efficiently identify a near-optimal hypothesis class. Our fixed budget algorithm uses a novel application of a selection-validation trick in bandits. This provides a new method for the understudied fixed budget setting in linear bandits (even without the added challenge of model selection). We further generalize the model selection problem to the misspecified regime, adapting our algorithms in both fixed confidence and fixed budget settings.

show abstract

Pareto Optimal Model Selection in Linear Bandits

Cited by 4 publications

References 11 publications

The Pareto Frontier of model selection for general Contextual Bandits

The Pareto Frontier of model selection for general Contextual Bandits

Universal and data-adaptive algorithms for model selection in linear contextual bandits

Near Instance Optimal Model Selection for Pure Exploration Linear Bandits

Contact Info

Product

Resources

About