Sequential Design and Analysis of High-Accuracy and Low-Accuracy Computer Codes

Abstract

A growing trend in engineering and science is to use multiple computer codes with different levels of accuracy to study the same complex system. We propose a framework for sequential design and analysis of a pair of high-accuracy and low-accuracy computer codes. It first runs the two codes with a pair of nested Latin hypercube designs (NLHDs). Data from the initial experiment are used to fit a prediction model. If the accuracy of the fitted model is less than a prespecified threshold, the two codes are evaluated again with input values chosen in an elaborate fashion so that their expanded scenario sets still form a pair of NLHDs. The nested relationship between the two scenario sets makes it easier to model and calibrate the difference between the two sources. If necessary, this augmentation process can be repeated a number of times until the prediction model based on all available data has reasonable accuracy. The effectiveness of the proposed method is illustrated with several examples. Matlab codes are provided in the online supplement to this article.

Modulus prediction of buckypaper based on multi-fidelity analysis involving latent variables

“…Not much has yet been published on a specific guideline or quantitative measure. One work that alludes to this aspect is Xiong et al (2013), which sets a threshold on testing the cross-validation error for continuation in a sequential design. Using this cross-validation measure does shed light on how a multi-fidelity model improves the predictive outcome, but one would still not know whether a multi-fidelity design is worth it or not until the crossvalidation error is computed (which has to be done after the multi-fidelity model is established).…”

“…In multi-fidelity analysis, one may be provided with data sets created by a physical experiment and a simulation model, such as in the aforementioned buckypaper fabrication process as well as in O'Hagan (2000, 2001), Higdon et al (2004), Reese et al (2004), Bayarri et al (2007), Qian and Wu (2008), Han et al (2009), andMelkote (2009), or they can come from two physical processes of different measurement resolutions (Xia et al, 2011) or from two simulation models of different degrees of accuracy (Qian et al, 2006;Xiong et al, 2013). Regardless of the origin of the data, in all of these cases one deals with a situation in which one experiment provides more accurate data (high fidelity) but obtained at a relatively higher cost, and the other experiment, despite being affordable, cannot be relied on solely as the responses or outputs do not reflect the reality very well (low fidelity).…”

Multifidelity surrogate modeling based on radial basis functions

“…The problems have been used by Xiong et al (2013) for illustration on multifidelity sequential design. The first one is a simple two-dimensional problem where the basic co-RBF prediction capability is tested.…”

“…The second test problem is based on the borehole model (Morris et al 1993) and is derived by Xiong et al (2013) as a multi-fidelity model. It describes the flow of water through a borehole drilled from the ground surface through two aquifers.…”

Prediction for Big Data Through Kriging: Small Sequential and One-Shot Designs

“…An analytical model of a borehole is used in many publications on simulation methodology; e.g., Erickson et al (2017), Gramacy (2016), Gramacy andApley (2015), andSantner et al (2018. p. 222). This model has the output "water ‡ow rate" w Table 3 (the range of one of these inputs may be changed to create a more nonlinear and non-additive function, as Xiong et al (2013) does):…”