Adaptive Policies for Sequential Sampling under Incomplete Information and a Cost Constraint

Burnetas, Apostolos; Kanavetas, Odysseas

doi:10.1007/978-1-4614-4109-0_8

Cited by 3 publications

(3 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…, k, where the randomization probabilities x j are an optimal solution to the above linear program LP(θ), cf. Burnetas and Kanavetas [7], Burnetas and Katehakis [12]. However, such policy may not be feasible in our framework that requires C π (n)/n ≤ c 0 , ∀n = 1, 2, .…”

Section: Preliminariesmentioning

confidence: 99%

“…Tran-Thanh et al [47], considered the problem when the cost of activation of each arm is fixed and becomes known after the arm is used once. Burnetas and Kanavetas [7] considered a version of this problem and constructed a consistent policy (i.e., with regret R π (n) = o(n), as n → ∞). In the present paper, we employ a stricter version of the average cost constraint that requires the average sampling cost not to exceed c 0 at any time period and not only in the limit.…”

Section: (13)mentioning

confidence: 99%

See 1 more Smart Citation

Asymptotically Optimal Multi-Armed Bandit Policies Under a Cost Constraint

Burnetas¹,

Kanavetas

Katehakis

2016

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

We develop asymptotically optimal policies for the multi armed bandit (MAB), problem, under a cost constraint. This model is applicable in situations where each sample (or activation) from a population (bandit) incurs a known bandit dependent cost. Successive samples from each population are iid random variables with unknown distribution. The objective is to design a feasible policy for deciding from which population to sample from, so as to maximize the expected sum of outcomes of n total samples or equivalently to minimize the regret due to lack on information on sample distributions, For this problem we consider the class of feasible uniformly fast (f-UF) convergent policies, that satisfy the cost constraint sample-path wise. We first establish a necessary asymptotic lower bound for the rate of increase of the regret function of f-UF policies. Then we construct a class of f-UF policies and provide conditions under which they are asymptotically optimal within the class of f-UF policies, achieving this asymptotic lower bound. At the end we provide the explicit form of such policies for the case in which the unknown distributions are Normal with unknown means and known variances. . Asymptotic optimality, finite horizon regret bounds, and a solution to an open problem. optimal Bayesian sequential change detection and identification rules. -armed bandit with budget constraint and variable costs. In AAAI-13, pages 232-238, 2013.Eugene A Feinberg, Pavlo O Kasyanov, and Michael Z Zgurovsky. Convergence of value iterations for total-cost mdps and pomdps with general state and action sets.

show abstract

Section: Preliminariesmentioning

confidence: 99%

Section: (13)mentioning

confidence: 99%

Asymptotically Optimal Multi-Armed Bandit Policies Under a Cost Constraint

Burnetas¹,

Kanavetas

Katehakis

2016

Prob. Eng. Inf. Sci.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further, the class of block-UCB (b-UCB) feasible policies which are developed here and achieve the asymptotic lower bound in the regret have a simpler form and are easier to compute than those in Burnetas et al (2017). We also refer to Burnetas and Kanavetas (2012) where a consistent policy (i.e., with regret o(n)) for the case of a single linear constraint was constructed.…”

Section: Introductionmentioning

confidence: 99%

Optimal Data Driven Resource Allocation under Multi-Armed Bandit Observations

Burnetas,

Kanavetas,

Katehakis

2018

Preprint

Self Cite

View full text Add to dashboard Cite

This paper introduces the first asymptotically optimal strategy for a multi armed bandit (MAB) model under side constraints. The side constraints model situations in which bandit activations are limited by the availability of certain resources that are replenished at a constant rate. The main result involves the derivation of an asymptotic lower bound for the regret of feasible uniformly fast policies and the construction of policies that achieve this lower bound, under pertinent conditions. Further, we provide the explicit form of such policies for the case in which the unknown distributions are Normal with unknown means and known variances, for the case of Normal distributions with unknown means and unknown variances and for the case of arbitrary discrete distributions with finite support.

show abstract