Achieving the Pareto Frontier of Regret Minimization and Best Arm Identification in Multi-Armed Bandits

Zhong, Zixin; Cheung, Wang Chi; Tan, Vincent Y. F.

doi:10.48550/arxiv.2110.08627

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While with f (j) = 2 log(j) the policy correctly determined all b fastest workers in ten simulation runs, with f (j) = 2 log(j)μ min in one out of ten simulations the algorithm commits to a worker with a suboptimality gap of 0.1. This reflects the trade-off between the competing objectives of best arm identification and regret minimization discussed in [42]. However, since the fastest workers have been determined eventually with an accuracy of 99.5%, the proposed adapted confidence bound seems reasonable improve the convergence rate.…”

Section: Simulation Resultsmentioning

confidence: 87%

Efficient Distributed Machine Learning via Combinatorial Multi-Armed Bandits

Maximilian

Bitar

Wachter-Zeh

et al. 2022

2022 IEEE International Symposium on Information Theory (ISIT)

View full text Add to dashboard Cite

We consider the distributed stochastic gradient descent problem, where a main node distributes gradient calculations among n workers from which at most b ≤ n can be utilized in parallel. By assigning tasks to all the workers and waiting only for the k fastest ones, the main node can trade-off the error of the algorithm with its runtime by gradually increasing k as the algorithm evolves. However, this strategy, referred to as adaptive k-sync, can incur additional costs since it ignores the computational efforts of slow workers. We propose a cost-efficient scheme that assigns tasks only to k workers and gradually increases k. As the response times of the available workers are unknown to the main node a priori, we utilize a combinatorial multi-armed bandit model to learn which workers are the fastest while assigning gradient calculations, and to minimize the effect of slow workers. Assuming that the mean response times of the workers are independent and exponentially distributed with different means, we give empirical and theoretical guarantees on the regret of our strategy, i.e., the extra time spent to learn the mean response times of the workers. Compared to adaptive k-sync, our scheme achieves significantly lower errors with the same computational efforts while being inferior in terms of speed.

show abstract

Section: Simulation Resultsmentioning

confidence: 87%