Adaptation to the Range in $K$-Armed Bandits

Abstract

We consider stochastic bandit problems with K arms, each associated with a bounded distribution supported on the range [m, M ]. We do not assume that the range [m, M ] is known and show that there is a cost for learning this range. Indeed, a new trade-off between distribution-dependent and distribution-free regret bounds arises, which, for instance, prevents from simultaneously achieving the typical ln T and √T bounds. For instance, a √ T distribution-free regret bound may only be achieved if the distribution-dependent regret bounds are at least of order √T . We exhibit a strategy achieving the rates for regret indicated by the new trade-off.Preprint. Under review.

Adaptive Best-of-Both-Worlds Algorithm for Heavy-Tailed Multi-Armed Bandits

“…Adaptive algorithms: There is a rich literature in deriving algorithms adaptive to the loss sequences, for either full information setting (Luo & Schapire, 2015;Orabona & Pal, 2016), stochastic bandits (Garivier & Cappé, 2011;Lattimore, 2015) or adversarial bandits (Wei & Luo, 2018;Bubeck et al, 2019). There are also many algorithms that is adaptive to the loss range, so-called 'scale-free' algorithms (De Rooij et al, 2014;Orabona & Pál, 2018;Hadiji & Stoltz, 2020). However, as mentioned above, to our knowledge, our work is the first to adapt to heavy-tail parameters.…”