Technical note—Knowledge gradient for selection with covariates: Consistency and computation

Abstract

Knowledge gradient is a design principle for developing Bayesian sequential sampling policies to solve optimization problems. In this paper, we consider the ranking and selection problem in the presence of covariates, where the best alternative is not universal but depends on the covariates. In this context, we prove that under minimal assumptions, the sampling policy based on knowledge gradient is consistent, in the sense that following the policy the best alternative as a function of the covariates will be identified almost surely as the number of samples grows. We also propose a stochastic gradient ascent algorithm for computing the sampling policy and demonstrate its performance via numerical experiments.

Online Appendix for 'Efficient Learning for Selecting Top-m Context-Dependent Designs'

“…In a setting where the context is infinite, it is impossible to sample every context value. [31], [32] and [33] consider the problem from a Bayesian perspective, employ acquisition functions (e.g., expected improvement and knowledge gradient), which are myopic approximations of the optimal sampling, and derive sequential sampling policies to maximize the posterior probability of good selection under a fixed simulation budget constraint. None of the existing work considers the top-m context-dependent selection problem.…”

Ranking and selection for pairwise comparison

“…The OCBA approach allocates a fixed number of simulation replications sequentially based on the performance mean and variance of each design such that the probability of correctly selecting the best design can be maximized (Chen et al, 2000). The VIP approach adopts the Bayesian framework and determines the future simulation budget allocation by maximizing the value information such as expected improvement and knowledge gradient (Chick et al, 2010; Ding et al, 2022; Frazier et al, 2008; Peng & Fu, 2017; Ryzhov, 2016).…”