An Improved Batch Means Procedure for Simulation Output Analysis

Abstract

W e formulate and evaluate the Automated Simulation Analysis Procedure (ASAP), an algorithm for steady-state simulation output analysis based on the method of nonoverlapping batch means (NOBM). ASAP delivers a confidence interval for an expected response that is centered on the sample mean of a portion of a simulation-generated time series and satisfies a user-specified absolute or relative precision requirement. ASAP operates as follows: The batch size is progressively increased until either (a) the batch means pass the von Neumann test for independence, and then ASAP delivers a classical NOBM confidence interval; or (b) the batch means pass the Shapiro-Wilk test for multivariate normality, and then ASAP delivers a correlation-adjusted confidence interval. The latter adjustment is based on an inverted Cornish-Fisher expansion for the classical NOBM t-ratio, where the terms of the expansion are estimated via an autoregressive-moving average time series model of the batch means. After determining the batch size and confidence-interval type, ASAP sequentially increases the number of batches until the precision requirement is satisfied. An extensive experimental study demonstrates the performance improvements achieved by ASAP versus well-known batch means procedures, especially in confidence-interval coverage probability.

Finite-Sample Performance of Absolute Precision Stopping Rules

“…Stopping rules are typically incorporated in CIPs that seek to return valid confidence intervals for data that may be dependent or nonnormal (Chen and Kelton 2007, Hoad et al 2009, Steiger and Wilson 2002, Tafazzoli et al 2011. Sequential ad hoc stopping rules are sometimes tailored to a specific set of simulation test models to provide better results.…”

Cumulative mean bounds for quality control analysis

“…normal. For examples, see Steiger and Wilson (2002), or for one that uses standardized time series, see Alexopoulos, Goldsman, Tang, and Wilson (2013). Assuming the samples are obtained from an i.i.d.…”

Indirect cycle time quantile estimation using the Cornish–Fisher expansion

“…Additionally, any stationary process that is also ergodic meets the conditions, having sample moments that converge to the true population moments (Robinson, 1994). An example of such a process is a φ-mixing process, which is commonly used to characterize the output process from DES models; see Glynn and Iglehart (1990), Chen and Kelton (2000) and Steiger and Wilson (2002), for example. Consequently, the terms of the CFE are asymptotically consistent.…”