We develop a statistical approach for characterizing uncertainty in predictions that are made with the aid of a computer simulation model. Typically, the computer simulation code models a physical system and requires a set of inputs-some known and specified, others unknown. A limited amount of field data from the true physical system is available to inform us about the unknown inputs and also to inform us about the uncertainty that is associated with a simulationbased prediction. The approach given here allows for the following: • uncertainty regarding model inputs (i.e., calibration); • accounting for uncertainty due to limitations on the number of simulations that can be carried out; • discrepancy between the simulation code and the actual physical system; • uncertainty in the observation process that yields the actual field data on the true physical system. The resulting analysis yields predictions and their associated uncertainties while accounting for multiple sources of uncertainty. We use a Bayesian formulation and rely on Gaussian process models to model unknown functions of the model inputs. The estimation is carried out using a Markov chain Monte Carlo method. This methodology is applied to two examples: a charged particle accelerator and a spot welding process.
A key question in evaluation of computer models is Does the computer model
adequately represent reality? A six-step process for computer model validation
is set out in Bayarri et al. [Technometrics 49 (2007) 138--154] (and briefly
summarized below), based on comparison of computer model runs with field data
of the process being modeled. The methodology is particularly suited to
treating the major issues associated with the validation process: quantifying
multiple sources of error and uncertainty in computer models; combining
multiple sources of information; and being able to adapt to different, but
related scenarios. Two complications that frequently arise in practice are the
need to deal with highly irregular functional data and the need to acknowledge
and incorporate uncertainty in the inputs. We develop methodology to deal with
both complications. A key part of the approach utilizes a wavelet
representation of the functional data, applies a hierarchical version of the
scalar validation methodology to the wavelet coefficients, and transforms back,
to ultimately compare computer model output with field output. The generality
of the methodology is only limited by the capability of a combination of
computational tools and the appropriateness of decompositions of the sort
(wavelets) employed here. The methods and analyses we present are illustrated
with a test bed dynamic stress analysis for a particular engineering system.Comment: Published in at http://dx.doi.org/10.1214/009053607000000163 the
Annals of Statistics (http://www.imstat.org/aos/) by the Institute of
Mathematical Statistics (http://www.imstat.org
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.