Asynchronous Parallel Empirical Variance Guided Algorithms for the Thresholding Bandit Problem

Zhong, Jie; Huang, Yijun; Liu, Ji

doi:10.48550/arxiv.1704.04567

Cited by 3 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This variant of the multi-armed bandit problem was introduced by Locatelli et al (2016), who provided an algorithm for solving the problem with matching upper and lower bounds. Mukherjee et al (2017) and Zhong et al (2017) have since provided algorithmic extensions that incorporate variance estimates and provide guarantees in asynchronous settings.…”

Section: Thresholding Banditsmentioning

confidence: 99%

Thresholding Graph Bandits with GrAPL

LeJeune¹,

Dasarathy²,

Baraniuk³

2019

Preprint

View full text Add to dashboard Cite

In this paper, we introduce a new online decision making paradigm that we call Thresholding Graph Bandits. The main goal is to efficiently identify a subset of arms in a multi-armed bandit problem whose means are above a specified threshold. While traditionally in such problems, the arms are assumed to be independent, in our paradigm we further suppose that we have access to the similarity between the arms in the form of a graph, allowing us gain information about the arm means in fewer samples. Such settings play a key role in a wide range of modern decision making problems where rapid decisions need to be made in spite of the large number of options available at each time. We present GrAPL, a novel algorithm for the thresholding graph bandit problem. We demonstrate theoretically that this algorithm is effective in taking advantage of the graph structure when available and the reward function homophily (that strongly connected arms have similar rewards) when favorable. We confirm these theoretical findings via experiments on both synthetic and real data.

show abstract

Section: Thresholding Banditsmentioning

confidence: 99%

Thresholding Graph Bandits with GrAPL

LeJeune¹,

Dasarathy²,

Baraniuk³

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…To improve the efficiency of bandit algorithms, multiple agents can be employed and they can perform simultaneous investigation. Zhong et al [2017] designed an asynchronous parallel bandit algorithm that allows multiple agents working in parallel without waiting for each other. Both theoretical analysis and empirical studies validate that the nearly linear speedup can be achieved.…”

Section: Parallelization Bandit Algorithmsmentioning

confidence: 99%

AutoML from Service Provider's Perspective: Multi-device, Multi-tenant Model Selection with GP-EI

Karlaš

Zhong

Zhang

et al. 2018

Preprint

Self Cite

View full text Add to dashboard Cite

AutoML has become a popular service that is provided by most leading cloud service providers today. In this paper, we focus on the AutoML problem from the service provider's perspective, motivated by the following practical consideration: When an AutoML service needs to serve multiple users with multiple devices at the same time, how can we allocate these devices to users in an efficient way? We focus on GP-EI, one of the most popular algorithms for automatic model selection and hyperparameter tuning, used by systems such as Google Vizer. The technical contribution of this paper is the first multi-device, multi-tenant algorithm for GP-EI that is aware of multiple computation devices and multiple users sharing the same set of computation devices. Theoretically, given N users and M devices, we obtain a regret bound of O((MIU(T, K) + M) N 2 M ), where MIU(T, K) refers to the maximal incremental uncertainty up to time T for the covariance matrix K. Empirically, we evaluate our algorithm on two applications of automatic model selection, and show that our algorithm significantly outperforms the strategy of serving users independently. Moreover, when multiple computation devices are available, we achieve near-linear speedup when the number of users is much larger than the number of devices.

show abstract

“…Even-Dar et al [2002], Chen and Li [2015], Simchowitz et al [2017], Garivier and Kaufmann [2016] for papers in the related best arm identification and TOP-M setting 1 in the fixed confidence case. The fixed budget version of TBP was studied in Chen et al [2014], , Mukherjee et al [2017], Zhong et al [2017] -but also see e.g. , Audibert and Bubeck [2010], Gabillon et al [2012], for papers in the related best arm identification and TOP-M setting in the fixed budget case.…”

Section: Introductionmentioning

confidence: 99%

The Influence of Shape Constraints on the Thresholding Bandit Problem

Cheshire¹,

Ménard²,

Carpentier³

2020

Preprint

View full text Add to dashboard Cite

We investigate the stochastic Thresholding Bandit problem (TBP ) under several shape constraints. On top of (i) the vanilla, unstructured TBP , we consider the case where (ii) the sequence of arm's means (µ k ) k is monotonically increasing MTBP , (iii) the case where (µ k ) k is unimodal UTBP and (iv) the case where (µ k ) k is concave CTBP . In the TBP problem the aim is to output, at the end of the sequential game, the set of arms whose means are above a given threshold. The regret is the highest gap between a misclassified arm and the threshold. In the fixed budget setting, we provide problem independent minimax rates for the expected regret in all settings, as well as associated algorithms. We prove that the minimax rates for the regret are (i) log(K)K/T for TBP , (ii) log(K)/T for MTBP , (iii) K/T for UTBP and (iv) log log K/T for CTBP , where K is the number of arms and T is the budget. These rates demonstrate that the dependence on K of the minimax regret varies significantly depending on the shape constraint. This highlights the fact that the shape constraints modify fundamentally the nature of the TBP .

show abstract

Asynchronous Parallel Empirical Variance Guided Algorithms for the Thresholding Bandit Problem

Cited by 3 publications

References 12 publications

Thresholding Graph Bandits with GrAPL

Thresholding Graph Bandits with GrAPL

AutoML from Service Provider's Perspective: Multi-device, Multi-tenant Model Selection with GP-EI

The Influence of Shape Constraints on the Thresholding Bandit Problem

Contact Info

Product

Resources

About