Selecting the best system and multi-armed bandits

Abstract

Consider the problem of finding a population or a probability distribution amongst many with the largest mean when these means are unknown but population samples can be simulated or otherwise generated. Typically, by selecting largest sample mean population, it can be shown that false selection probability decays at an exponential rate. Lately, researchers have sought algorithms that guarantee that this probability is restricted to a small δ in order log(1/δ) computational time by estimating the associated large deviations rate function via simulation. We show that such guarantees are misleading when populations have unbounded support even when these may be light-tailed. Specifically, we show that any policy that identifies the correct population with probability at least 1 − δ for each problem instance requires infinite number of samples in expectation in making such a determination in any problem instance. This suggests that some restrictions are essential on populations to devise O(log(1/δ)) algorithms with 1 − δ correctness guarantees. We note that under restriction on population moments, such methods are easily designed, and that sequential methods from stochastic multi-armed bandit literature can be adapted to devise such algorithms.

Statistically Robust, Risk-Averse Best Arm Identification in Multi-Armed Bandits

“…The remainder of this section is devoted to the proof of Theorem 2. We note here that a similar impossibility result was proved by [25] for the pure exploration bandit problem in the fixed-confidence setting. Our proof technique is inspired their methodology and also relies crucially on the lower bounds in [24].…”

Optimal Allocations for Sample Average Approximation

“…First, we assume that the (rate) function J(α x , α y ,t) can be analytically optimized over t. In this case, the objective in ( 9) is simpler to estimate and optimize. Note that these instances are direct analogues in the SAA context of the ranking and selection (R&S) problems studied in [4,7], and [6]. To deal with these types of problems, we present Algorithm 1 below, that parallels [7] Algorithm 2.…”

Data-driven R&S under Input Uncertainty

“…While normality is justified by batching and the Central Limit Theorem (CLT), the assumption typically fails when simulations are run under an estimate of θ c , especially if such an estimate is updated in an online fashion. In this paper, we build our procedures on a Sequential Elimination framework ( [11,12,13]), as it allows us to construct valid continuation regions even in the presence of IU. Here we use a production-inventory problem (see Section 5 for details) to illustrate how our procedure works.…”

“…Our first procedure is a direct extension of a Sequential Elimination framework proposed by [11,12], which is also discussed in [13] recently. This general paradigm has a simple structure and can be extended to handle IU.…”

“…In particular, we find two general frameworks the most relevant to us. One is a Sequential Elimination framework for fixed confidence ( [11,12,13]), and the other is the Optimal Computing Budget Allocation (OCBA) framework for fixed budget ( [14,15]). A more detailed introduction to these frameworks will be provided later on, along with a discussion on why other methods do not conveniently apply to the settings considered in this paper.…”