The goal of ranking and selection (R&S) procedures is to identify the best stochastic system from among a finite set of competing alternatives. Such procedures require constructing estimates of each system's performance, which can be obtained simultaneously by running multiple independent replications on a parallel computing platform. However, nontrivial statistical and implementation issues arise when designing R&S procedures for a parallel computing environment. Thus we propose several design principles for parallel R&S procedures that preserve statistical validity and maximize core utilization, especially when large numbers of alternatives or cores are involved. These principles are followed closely by our parallel Good Selection Procedure (GSP), which, under the assumption of normally distributed output, (i) guarantees to select a system in the indifference zone with high probability, (ii) runs efficiently on up to 1,024 parallel cores, and (iii) in an example uses smaller sample sizes compared to existing parallel procedures, particularly for large problems (over 10 6 alternatives). In our computational study we discuss two methods for implementing GSP on parallel computers, namely the Message-Passing Interface (MPI) and Hadoop MapReduce and show that the latter provides good protection against core failures at the expense of a significant drop in utilization due to periodic unavoidable synchronization.(Henderson and Pasupathy 2014). For overviews of methods to solve the SO problem, see, e.g., Fu (1994), Andradóttir (1998), Fu et al. (2005, Pasupathy and Ghosh (2013).We consider the case of SO on finite sets, in which the decision variables can be categorical, integer-ordered and finite, or a finite "grid" constructed from a continuous space. Formally, the SO problem on finite sets can be written aswhere S = {1, . . . , k} is a finite set of design points or "systems" indexed by i, and ξ is a random element used to model the stochastic nature of simulation experiments. (In the remainder of the paper we assume that µ 1 ≤ µ 2 ≤ · · · ≤ µ k , so that system k is the best.) The objective function µ : S → R cannot be computed exactly, but can be estimated using output from a stochastic simulation represented by X(·; ξ). While the feasible space S may have topology, as in the finite but integer-ordered case, we consider only methods to solve the SO problem in (1) that (i) do not exploit such topology or structural properties of the function, and that (ii) apply when the computational budget permits at least some simulation of every system. Such methods are called ranking and selection (R&S) procedures.R&S procedures are frequently used in simulation studies because structural properties, such as convexity, are difficult to verify for simulation models and rarely hold. They can also be used in conjunction with heuristic search procedures in a variety of ways (Pichitlamken et al. 2006, Boesel et al. 2003, making them useful even if not all systems can be simulated. See Kim and Nelson (2006a) for an excellent introdu...

Ambulance redeployment is the practice of repositioning ambulance fleets in real time in an attempt to reduce response times to future calls. When redeployment decisions are based on real-time information on the status and location of ambulances, the process is called system-status management. An important performance measure is the long-run fraction of calls with response times over some time threshold. We construct a lower bound on this performance measure that holds for nearly any ambulance redeployment policy through comparison methods for queues. The computation of the bound involves solving a number of integer programs and then simulating a multiserver queue. This work originated when one of the authors was asked to analyze a response to a request-for-proposals (RFP) for ambulance services in a county in North America.

We explore the adaptation of a ranking and selection procedure, originally designed for a sequential computer, to a high-performance (parallel) computing setting. We pay particular attention to screening and explaining why care is required in implementing screening in parallel settings. We develop an algorithm that allows screening at both the master and worker levels, and that apportions work to processors in such a way that excessive communication is avoided. In doing so we rely on a random number generator with many streams and substreams. 833978-1-4799-3950-3/13/$31.00 ©2013 IEEE Ni, Hunter, and Henderson the reported system is, indeed, best. We limit ourselves to R&S algorithms, rather than to general SO algorithms (that focus on search and do not provide any kind of statistical guarantees on the quality of a reported solution) because the interplay of the desire for statistical guarantees along with the desire for computational efficiency creates interesting algorithmic tension and challenges.We develop parallel R&S algorithms in the context of HPC environments, which can include thousands of cores. The HPC environment differs from other, less-reliable parallel architectures in that cores rarely fail, communication is fast and relatively straightforward, and memory is not typically shared between cores. These features mean that HPC environments are perhaps the most straightforward of parallel environments in which to explore stochastic simulation optimization. Further, while HPC environments may be lessavailable to the general public than cloud computing environments such as Amazon, our work is supported by the Extreme Science and Engineering Discovery Environment (XSEDE), an NSF-funded infrastructure for high-performance computing available to researchers worldwide. Thus access to an HPC environment is a plausible reality for many researchers.The reliability of the HPC environment also enables us to develop a relatively simple structure for our parallel algorithms, in which one core is dedicated as the "master" and all other cores are "workers." The master assigns tasks to the workers, which the workers complete in parallel. The master also operates as a coordinator for the workers, so that each worker only communicates with the master and not with the other workers. To be available to process tasks from the workers, the master does not perform any simulation replications. In other less-reliable computing environments, this structure may not be desirable since cores -in particular, the master -may fail. However in the HPC environment, this simple structure is stable. We adopt a master-worker framework for the algorithms we develop.Our study has similarities with Luo and Hong (2011), and indeed, that paper helped to motivate our work. Both studies involve the implementation of ranking and selection algorithms in parallel computing environments, and both employ sequential screening methods. An important difference is that we are working in a high-performance environment that affords the advantages mentioned ...

Traditional solutions to ranking and selection problems include two-stage procedures (e.g., the NSGS procedure of Nelson et al. 2001) and fully-sequential screening procedures (e.g., Hong 2006). In a parallel computing environment, a naively-parallelized NSGS procedure may require more simulation replications than a sequential screening procedure such as that of Ni, Hunter, and Henderson (2013) (NHH), but requires less communication since there is no periodic screening. The parallel procedure NHH may require less simulation replications overall, but requires more communication to implement periodic screening. We numerically explore the trade-offs between these two procedures on a parallel computing platform. In particular, we discuss their statistical validity, efficiency, and implementation, including communication and load-balancing. Inspired by the comparison results, we propose a framework for hybrid procedures that may further reduce simulation cost or guarantee to select a good system when multiple systems are clustered near the best.

Some queueing systems require tremendously long simulation runlengths to obtain accurate estimators of certain steady-state performance measures when the servers are heavily utilized. However, this is not uniformly the case. We analyze a number of single-station Markovian queueing models, demonstrating that several steady-state performance measures can be accurately estimated with modest runlengths. Our analysis reinforces the meta result that if the queue is “well dimensioned,” then simulation runlengths will be modest. Queueing systems can be well dimensioned because customers abandon if they are forced to wait in line too long, or because the queue is operated in the “quality- and efficiency-driven regime” in which servers are heavily utilized but wait times are short. The results are based on computing or bounding the asymptotic variance and bias for several standard single-station queueing models and performance measures.