High-Dimensional Experimental Design and Kernel Bandits

Romain, Camilleri,; Katz-Samuels, Julian; Jamieson, Kevin

doi:10.48550/arxiv.2105.05806

Cited by 3 publications

(6 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark 4. Our sample complexity guarantees are directly related to the eigenvalue decay of the underlying kernel function, rather than the empirical kernel matrix as studied in previous works [7,42]. Although one can also provide an instance dependent bound as in Theorem 1, the worst-case sample complexity bound in Theorem 2 provides insightful characterizations of the sample complexity in terms of eigenvalue decay.…”

Section: Pure Exploration In Rkhsmentioning

confidence: 96%

“…Learning with model misspecifications was recently introduced in bandit learning, with the primary emphasis placed on the regret minimization problem [16,28,14]. A very recent independent work studies pure exploration in kernel bandits with misspecifications [7]. They propose a robust estimator that works in high-dimensional spaces and also explore the idea of low-dimensional embeddings through regularized least squares.…”

Section: Related Workmentioning

confidence: 99%

“…User Click Log Dataset R6A in this experiment. 7 The dataset contains the users' click-through records from the Today news module on Yahoo! front page between May 1st.…”

Section: G2 Detailed Descriptions Of Datasetsmentioning

confidence: 99%

“…To overcome the curse of dimensionality, we propose to adaptively embed each arm's feature representation in a lower-dimensional space and carefully deal with the induced misspecifications. Note that our approach is conceptually very different from all existing work dealing with model misspecifications: they assume the existence of misspecifications and address it the original space (thus dealing with model misspecifications in a passive way) [28,7]. On the other hand, we deliberately induce (acceptable) misspecifications to embed arms into lower-dimensional spaces and thus overcome the curse of dimensionality.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Pure Exploration in Kernel and Neural Bandits

Zhu¹,

Zhou²,

Jiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

We study pure exploration in bandits, where the dimension of the feature representation can be much larger than the number of arms. To overcome the curse of dimensionality, we propose to adaptively embed the feature representation of each arm into a lower-dimensional space and carefully deal with the induced model misspecifications. Our approach is conceptually very different from existing works that can either only handle low-dimensional linear bandits or passively deal with model misspecifications. We showcase the application of our approach to two pure exploration settings that were previously under-studied: (1) the reward function belongs to a possibly infinite-dimensional Reproducing Kernel Hilbert Space, and (2) the reward function is nonlinear and can be approximated by neural networks. Our main results provide sample complexity guarantees that only depend on the effective dimension of the feature spaces in the kernel or neural representations. Extensive experiments conducted on both synthetic and real-world datasets demonstrate the efficacy of our methods.

show abstract

Section: Pure Exploration In Rkhsmentioning

confidence: 96%

Section: Related Workmentioning

confidence: 99%

“…User Click Log Dataset R6A in this experiment. 7 The dataset contains the users' click-through records from the Today news module on Yahoo! front page between May 1st.…”

Section: G2 Detailed Descriptions Of Datasetsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Pure Exploration in Kernel and Neural Bandits

Zhu¹,

Zhou²,

Jiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We aim to provide sample complexity/error probability guarantees with respect to a hypothesis class that is rich enough to allow us to identify an -optimal arm. [3,22] recently study pure exploration with misspecifications in the fixed confidence setting; we additionally provide the first analysis for the fixed budget setting. The model selection criterion also complicates the analysis and is not covered in previous work.…”

Section: Model Selection With Misspecificationsmentioning

confidence: 99%

Near Instance Optimal Model Selection for Pure Exploration Linear Bandits

Zhu¹,

Katz-Samuels²,

Nowak³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The model selection problem in the pure exploration linear bandit setting is introduced and studied in both the fixed confidence and fixed budget settings. The model selection problem considers a nested sequence of hypothesis classes of increasing complexities. Our goal is to automatically adapt to the instance-dependent complexity measure of the smallest hypothesis class containing the true model, rather than suffering from the complexity measure related to the largest hypothesis class. We provide evidence showing that a standard doubling trick over dimension fails to achieve the optimal instance-dependent sample complexity. Our algorithms define a new optimization problem based on experimental design that leverages the geometry of the action set to efficiently identify a near-optimal hypothesis class. Our fixed budget algorithm uses a novel application of a selection-validation trick in bandits. This provides a new method for the understudied fixed budget setting in linear bandits (even without the added challenge of model selection). We further generalize the model selection problem to the misspecified regime, adapting our algorithms in both fixed confidence and fixed budget settings.

show abstract

An Experimental Design Approach for Regret Minimization in Logistic Bandits

Mason

Jun

Jain

2022

AAAI

View full text Add to dashboard Cite

In this work we consider the problem of regret minimization for logistic bandits. The main challenge of logistic bandits is reducing the dependence on a potentially large problem dependent constant that can at worst scale exponentially with the norm of the unknown parameter vector. Previous works have applied self-concordance of the logistic function to remove this worst-case dependence providing regret guarantees that move the reduce the dependence on this worst case parameter to lower order terms with only polylogarithmic dependence on the main term and as well as linear dependence on the dimension of the unknown parameter. This work improves upon the prior art by 1) removing all scaling of the worst case term on the main term and 2) reducing the dependence on the dependence to scale with the square root of dimension in the fixed arm setting by employing an experimental design procedure. Our regret bound in fact takes a tighter instance (i.e., gap) dependent regret bound for the first time in logistic bandits. We also propose a new warmup sampling algorithm that can dramatically reduce the lower order term in the regret in general and prove that it can exponentially reduce the lower order term's dependency on the worst case parameter in some instances. Finally, we discuss the impact of the bias of the MLE on the logistic bandit problem in d dimensions, providing an example where d^2 lower order regret (cf., it is d for linear bandits) may not be improved as long as the MLE is used and how bias-corrected estimators may be used to make it closer to d.

show abstract

High-Dimensional Experimental Design and Kernel Bandits

Cited by 3 publications

References 3 publications

Pure Exploration in Kernel and Neural Bandits

Pure Exploration in Kernel and Neural Bandits

Near Instance Optimal Model Selection for Pure Exploration Linear Bandits

An Experimental Design Approach for Regret Minimization in Logistic Bandits

Contact Info

Product

Resources

About