Using Cross Validation to Design Conservative Surrogates

Abstract

The use of surrogates (also known as metamodels) for facilitating optimization and statistical analysis of computationally expensive simulations has become commonplace. Surrogate models are usually fit to be unbiased (i.e., the error expectation is zero). However, in certain applications, it might be important to safely estimate the response (e.g., in structural analysis, the maximum stress must not be underestimated in order to avoid failure). In this work we use safety margins to conservatively compensate for fitting errors associated with surrogates. We propose the use of cross validation for estimating the required safety margin for a desired level of conservativeness (percentage of safe predictions). The approach was tested on three algebraic examples for two basic surrogates: namely, kriging and polynomial response surface. For these examples we found that cross validation is effective for selecting the safety margin. We also applied the approach to the probabilistic design optimization of a composite laminate. This design under uncertainty example showed that the approach can be successfully used in engineering applications.

Review of surrogate modeling in water resources

“…Although such a trial and error approach to penalty function parameters can be difficult to avoid, such experimentation with a metamodel‐enabled optimizer will present incredible computational challenges. There are also different approaches in the broader research community to more accurately handle constraints with surrogates [e.g., Kim and Lee , 2010; Lee et al , 2007; Picheny et al , 2008; Viana et al , 2010]. The paper by Viana et al [2010] nicely overviews these general approaches (designing conservative surrogates and adaptively improving surrogate accuracy near the boundary between feasible and infeasible solutions).…”

An algorithm for fast optimal Latin hypercube design of experiments

“…All simulations were conducted using an Intel Core 2 Quad CPU Q6600 at 2.40 GHz, with 3 GB of RAM running MATLAB 7.6 (R2008a) under Windows XP. The SURROGATES toolbox [30] was used to execute the TPLHD, ESEA, and GA algorithms under MATLAB [29].…”

Doubly weighted moving least squares and its application to structural reliability analysis

“…Queipo et al 2005;Viana et al 2010b). In conventional approximation problems, the general criterion of a surrogate model is to minimize the error between the true function and the surrogate model in the entire domain of interest.…”