Skart: A skewness- and autoregression-adjusted batch-means procedure for simulation analysis

Abstract

We discuss Skart, an automated batch-means procedure for constructing a skewness-and autoregression-adjusted confidence interval for the steady-state mean of a simulation output process. Skart is a sequential procedure designed to deliver a confidence interval that satisfies user-specified requirements concerning not only coverage probability but also the absolute or relative precision provided by the half-length. Skart exploits separate adjustments to the half-length of the classical batch-means confidence interval so as to account for the effects on the distribution of the underlying Student's t-statistic that arise from nonnormality and autocorrelation of the batch means. Skart also delivers a point estimator for the steady-state mean that is approximately free of initialization bias. In an experimental performance evaluation involving a wide range of test processes, Skart compared favorably with other simulation analysis methods-namely, its predecessors ASAP3, WASSP, and SBatch as well as ABATCH, LBATCH, the Heidelberger-Welch procedure, and the Law-Carson procedure. 978-1-4244-2708-6/08/$25.00 THE METHOD OF BATCH MEANSIn the method of nonoverlapping batch means (NBM), the outputs fX i W i D 1; : : : ng are divided into k adjacent nonoverlapping batches, each of size m, where we assume n D km and both k and m are sufficiently large to ensure that the resulting batch means are at least approximately independent and identically distributed (i.i.d.) normal random variables. The sample mean for the j th batch is AUTHOR BIOGRAPHIES

Reviving the Two-State Markov Chain Approach

“…We implemented the two-state Markov chain approach with the simple heuristics presented in Section 3 and the Skart method of [11] in the tool ASSA-PBN, which was specially designed for steady-state analysis of large PBNs [19] (see Section 2.2 for the theoretical background of PBNs). We verified with experiments that with use of the simple heuristics, the two-state Markov chain approach could meet the predefined precision requirement even in the case of an unlucky initial sample size.…”

“…We choose the Skart method [11] as a reference for the evaluation of the performance of the two-state Markov chain approach. The Skart method is a successor of ASAP3, WASSP, and SBatch methods, which are all based on the idea of batch means [11].…”

“…We choose the Skart method [11] as a reference for the evaluation of the performance of the two-state Markov chain approach. The Skart method is a successor of ASAP3, WASSP, and SBatch methods, which are all based on the idea of batch means [11]. It is a procedure for on-the-fly statistical analysis of the simulation output, asymptotically generated in accordance with a steady-state distribution.…”

“…A number of statistical methods exists, which allow to empirically determine when to stop the simulation and output estimates. We employ in our study: (1) the two-state Markov chain approach [10] and (2) the Skart batch-means method of [11]. The two-state Markov chain approach was introduced in 1992 by Raftery and Lewis; and Shmulevich et al [8] proposed its application to the analysis of PBNs in 2003.…”

Displaying statistical point estimates using the leading-digit rule

“…Even when the user carefully interprets the interval, violations of the normality and independence assumptions cause the true coverage probability to differ from the nominal value. This difference is the motivation of many research papers (e.g., Schmeiser (1982Schmeiser ( , 1992; Song and Schmeiser (1993); Song (1996); Alexopoulos and Goldsman (2004); and Tafazzoli et al (2008), that discuss, propose, and evaluate CIPs for various simulation contexts, especially that of stationary autocorrelated data. 3.…”