Variance reduction techniques: Experimental comparison and analysis for single systems

Abstract

We provide a thorough analysis of the effectiveness of different Variance Reduction Techniques (VRTs). We consider both stand-alone and combined applications of two input techniques, Antithetic Variates (AV) and Latin Hypercube Sampling (LHS), and two output techniques, Control Variates (CV) and Poststratified Sampling (PS). Previous research in the area mainly focuses on asymptotic variance reduction. In this experimental study, we measure the performance of VRTs under finite simulation run lengths and analyze their effects. Our findings show that the asymptotic variance reduction results do not readily apply to finite-length simulations. We consider three different types of systems (M/M/1, serial production line and (s, S) inventory control systems) and compare the VRTs under various experimental conditions. We observe that a variance reduction cannot be guaranteed for every instance a VRT is applied. Our results also indicate that the output VRTs (CV, PS) are better than input VRTs (AV, LHS) on the average for the single systems considered in this study. More interestingly, the less-sophisticated techniques (AV, CV) often perform better than the relatively more-complex techniques (LHS, PS). A comprehensive bibliography is also provided

Combined variance reduction techniques in fully sequential selection procedures

“…When applying the stratified sampling technique, we must know the exact distribution of the stratification variable, which is usually assumed to be normally distributed, at least asymptotically, in the related literature. For the determination of the boundaries between the strata, the experimental results from existing literature often reveal that Sethi's stratification scheme is better than the equal‐probability scheme (eg, Sabuncuoglu et al, ). Therefore, in the current experiments we use Sethi's optimal stratification scheme for a normal random variable under proportional allocation to classify each of the replicated observations into the appropriate stratum.…”

“…Therefore, in the current experiments we use Sethi's optimal stratification scheme for a normal random variable under proportional allocation to classify each of the replicated observations into the appropriate stratum. In the literature on PS technique, is often specified in the range of , where it is experimentally found that the marginal efficiency gain may not be significant when using more than strata (see Chapter 5 of Cochran (), Sabuncuoglu et al (), and Wilson and Pritsker ()). In the following experimental study, we use and .…”

“…The combined scheme with CV and stratified sampling (SS) was studied by L'Ecuyer and Buist () within the content of a call center simulation model, where stratification was applied to uniform random numbers driving the simulation. Sabuncuoglu, Fadiloglu, and Çelik () investigated the experimental performance of individual use of different VRTs, including CV, AV, PS, and Latin hypercube sampling (LHS), as well as the combined CV + AV and CV + LHS schemes. They considered three classical simulation models, that is, queueing, a serial production line, and inventory control systems, and the experimental results (for the single use of VRT) showed that CV exhibits the best performance followed by PS.…”

“…Different from Tsai and Kuo (), in this paper, we consider combined models of CV with output techniques (ie, CV + CE and CV + PS), and design new fully sequential procedures when combined models may sometimes not be applicable (which motivates us to use CV + PCE). This research work is also motivated by the observation that output techniques can often obtain more statistical efficiency than input techniques, as shown in the related literature (eg, Avramidis & Wilson, ; Grant ; Sabuncuoglu et al, ). We also note that the aforementioned CV + PS scheme in the literature assumes a fixed sample size and requires implementing the regression for each stratum.…”

Improving Precision and Reducing Runtime of Microscopic Traffic Simulators through Stratified Sampling

“…For the case of a queuing system with a single server, Page [27] and Mitchell [28] applied AV to the generation of exponentially distributed customer inter-arrival and service times, noting reductions of up to 41% in the standard error of the estimated average customer waiting time when compared to simulation by SRS. Sabuncuoglu et al [29] experimented with AV and LHS in turn for the same queuing system, observing reductions of up to 16% in the standard error of the estimator of the total time spent in the system relative to SRS. Ross and Lin [30] advocate stratification of the number of customer arrivals in queuing systems as opposed to inter-arrival times and obtained a reduction in the standard deviation of the average customer waiting time of 29%.…”