Efficient Simulation Budget Allocation for Selecting an Optimal Subset

Chen, Chun‐Hung; He, Donghai; Fu, Michael C.; Lee, Loo Hay

doi:10.1287/ijoc.1080.0268

Cited by 209 publications

(115 citation statements)

References 25 publications

Supporting

Mentioning

115

Contrasting

Order By: Relevance

“…The allocation may be between exploration of different designs and estimation of objective function values at specific designs as in global optimization [10,14], between estimation of different random variables nested by conditioning [21], or between estimation of different expected system performances in ranking and selection [9]. These studies typically define an optimal allocation as one that makes the estimator mean-squared error vanish at the fastest possible rate as the computing budget tends to infinity.…”

Section: Introductionmentioning

confidence: 99%

Optimal Budget Allocation for Sample Average Approximation

Røyset

Szechtman

2013

Operations Research

View full text Add to dashboard Cite

Abstract. The sample average approximation approach to solving stochastic programs induces a sampling error, caused by replacing an expectation by a sample average, as well as an optimization error due to approximating the solution of the resulting sample average problem. We obtain estimators of an optimal solution and the optimal value of the original stochastic program after executing a finite number of iterations of an optimization algorithm applied to the sample average problem. We examine the convergence rate of the estimators as the computing budget tends to infinity, and characterize the allocation policies that maximize the convergence rate in the case of sublinear, linear, and superlinear convergence regimes for the optimization algorithm.

show abstract

Section: Introductionmentioning

confidence: 99%

Optimal Budget Allocation for Sample Average Approximation

Røyset

Szechtman

2013

Operations Research

View full text Add to dashboard Cite

show abstract

“…There are many variants of OCBA that consider correlated sampling [16]; non-normal distributions [17]; different objective functions [18]; subset selection [19]; complete ranking [20]; and constraints [21]. Lee et al [22] provided a comprehensive review of these OCBA procedures.…”

Section: 2mentioning

confidence: 99%

Single- And Multi-Objective Ranking and Selection Procedures in Simulation: A Historical Review

Yoon

Bekker

2017

SAJIE

View full text Add to dashboard Cite

Ranking and selection (R&S) procedures form an important research field in computer simulation and its applications. In simulation, one usually has to select the best from a number of scenarios or alternative designs. Often, the simulated processes have a stochastic nature, which means that, to distinguish alternatives, they must exhibit significant statistical differences. R&S procedures assist the decision-maker with the selection of the best alternative with high confidence. This paper reviews past and current R&S procedures. The review traces back to the 1950s, when the first R&S procedure was proposed, and discusses the various R&S procedures proposed since then to the present day, presenting a cursory view of the research in the area. The review includes studies in both the single-objective and the multi-objective domains. It presents the research trend, discusses specific issues, and gives recommendations for future research in both domains. OPSOMMINGRangorde-en keuseprosedures (R&K) vorm 'n belangrike navorsingsveld in rekenaarsimulasie en -toepassings. Simulasiestudies vereis gewoonlik dat die beste kandidaat van 'n aantal scenarios of alternatiewe ontwerpe gekies moet word. Die gesimuleerde prosesse is gewoonlik van 'n stogastiese aard, en hulle moet statisties-beduidend verskil ten einde onderskeid te kan tref. R&K prosedures ondersteun die besluitnemer om die beste alternatief met groot vertroue te kies. Hierdie artikel verskaf 'n resensie van vroeë en huidige R&K prosedures. Die ondersoek strek terug tot in die 1950s toe die eerste R&K prosedures voorgestel is, en bespreek die verskeie R&K prosedures wat sedertdien ontwikkel is, terwyl 'n oorsigtelike blik op die navorsingsveld gegee word. Die resensie sluit studies in beide enkel-en multidoelwitdomein in. Dit bespreek navorsingsneigings en spesifieke kwessies, en maak voorstelle vir verdere navorsing in beide domeins. INTRODUCTIONComputer simulation is a powerful and essential tool in modern society to improve the operation of current systems and business practices. It contributes to the efficient management of processes and systems -the main focus of many industrial engineers -by providing a what-if analysis. In operating a system, one often faces a situation where the following question arises: If some changes were made to the current parameter settings (or decision variables) 1 in the operation of the system, what would happen? Would they result in a better performance of the system or not? It is usually not easy to estimate the effect of the changes because the system is often complex and exhibits a stochastic nature. Simulation helps decision-makers in such cases by simulating the real system and providing the estimates of system performance measures. One can compare the performance of the simulated 1 Various terms are used in simulation to indicate parameter settings of the system: system designs, systems, designs, scenarios, and/or alternatives. The term population is also used in the same context in statistics.

show abstract

“…One is to maximize the probability of choosing the top m systems (Chen, He, Fu, and Lee 2008), while others wish to choose a subset of size m that contains some number of the best systems (Koenig and Law 1985). In Ryzhov and Powell (2009), possible subsets are treated as individual alternatives and sampling rules applied to choose the best subset.…”

Section: Literature Reviewmentioning

confidence: 99%

Multiple comparisons with a standard using false discovery rates

Singham¹,

Szechtman²

2016

2016 Winter Simulation Conference (WSC)

View full text Add to dashboard Cite

Multiple comparisons with a standard using false discovery rates Downloaded from NPS Archive: CalhounProceedings of the 2016 Winter Simulation Conference T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick, eds. MULTIPLE COMPARISONS WITH A STANDARD USING FALSE DISCOVERY RATES ABSTRACTWe introduce a new framework for performing multiple comparisons with a standard when simulation models are available to estimate the performance of many different systems. In this setting, a large proportion of the systems have mean performance from some known null distribution, and the goal is to select alternative systems whose means are different from that of the null distribution. We employ empirical Bayes ideas to achieve a bound on the false discovery rate (proportion of selected systems from the null distribution) and a desired probability an alternate type system is selected.

show abstract

Efficient Simulation Budget Allocation for Selecting an Optimal Subset

Cited by 209 publications

References 25 publications

Optimal Budget Allocation for Sample Average Approximation

Optimal Budget Allocation for Sample Average Approximation

Single- And Multi-Objective Ranking and Selection Procedures in Simulation: A Historical Review

Multiple comparisons with a standard using false discovery rates

Contact Info

Product

Resources

About