Observe Before Play: Multi-armed Bandit with Pre-observations

Zuo, Jinhang; Zhang, Xiaoxi; Joe‐Wong, Carlee

doi:10.48550/arxiv.1911.09458

Cited by 1 publication

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a recent work [10], the authors study a bandit problem with Bernoulli rewards where in each round, the decision maker proposes an ordered list of the K arms, and plays the first arm with observed reward equal to 1. This problem is similar to that investigated in [11].…”

Section: Stochastic Bandit Problems Have Been Extensively Studiedmentioning

confidence: 99%

See 1 more Smart Citation

Predictive Bandits

Lindståhl,

Proutiere,

Johnsson

2020

Preprint

View full text Add to dashboard Cite

We introduce and study a new class of stochastic bandit problems, referred to as predictive bandits. In each round, the decision maker first decides whether to gather information about the rewards of particular arms (so that their rewards in this round can be predicted). These measurements are costly, and may be corrupted by noise. The decision maker then selects an arm to be actually played in the round. Predictive bandits find applications in many areas; e.g. they can be applied to channel selection problems in radio communication systems. In this paper, we provide the first theoretical results about predictive bandits, and focus on scenarios where the decision maker is allowed to measure at most one arm per round. We derive asymptotic instance-specific regret lower bounds for these problems, and develop algorithms whose regret match these fundamental limits. We illustrate the performance of our algorithms through numerical experiments. In particular, we highlight the gains that can be achieved by using reward predictions, and investigate the impact of the noise in the corresponding measurements.

show abstract

Section: Stochastic Bandit Problems Have Been Extensively Studiedmentioning

confidence: 99%

“…). (10) This implies that an asymptotic lower bound for the regret is C(θ) log(T ), where C(θ) is the value of the solution of the following optimization problem.…”

Section: Appendix I Proof Of Theoremmentioning

confidence: 99%