Design and Analysis of Variable Fidelity Experimentation Applied to Engine Valve Heat Treatment Process Design

Abstract

When experimentation on a real system is expensive, data are often collected by using cheaper, lower fidelity surrogate systems. The paper concerns response surface methods in the context of variable fidelity experimentation. We propose the use of generalized least squares to generate the predictions. We also present perhaps the first optimal designs for variable fidelity experimentation, using an extension of the expected integrated mean-squared error criterion. Numerical tests are used to compare the performance of the method with alternatives and to investigate the robustness to incorporated assumptions. The method is applied to automotive engine valve heat treatment process design in which real world data were mixed with data from two types of computer simulation. Copyright 2005 Royal Statistical Society.

A Simple Approach for Multi‐fidelity Experimentation Applied to Financial Engineering

“…It can be seen that when designs address bias, such as those of Huang and Allen , the ‘shrunk’ MPCCD and our proposed design, they perform better than those models that do not consider bias at all, such as the Central Composite Design. This point is also made by Huang and Allen . In addition, in the first class and in Huang and Allen's research, the proposed relationship between each surrogate system is assumed to be true .…”

“…An area of increasing interest to researchers is the study of methods that exploit variable fidelity data, that is, measurements from multiple systems with varying costs that approximate results from a system of interest. Huang and Allen first proposed the optimal experimental design methods for the variable fidelity experimentation using the criterion of Expected Integrated Mean Squared Error (EIMSE). In order for this methodology to be utilized, a covariance matrix including systematic and random errors to fit a predictive model and generate an optimal design under cost constraints must be pre‐estimated before data are collected.…”

“…Huang and Allen's design using the Generalized Least Squares (GLS) analysis predictably gave the best performance compared to the alternatives that they considered. However, in actual practice, the researcher is unaware of the relationship between the fidelity surrogate systems.…”

“…Section 3 presents the EIMSE criterion and discusses how it can be extended for use in situations involving qualitative factors and the associated new class of EIMSE‐optimal response surface designs. Section 4 includes a numerical example taken from Huang and Allen , which provides a comparison of the proposed methods with alternatives from the literature. The application of the proposed methods to the financial engineering case study is described in Section 5.…”

Bayesian optimization for parameter identification on a small simulation budget

“…The field of optimization techniques that rely on such models is known as Bayesian Optimization [Zilinskas, 1992]. The interest of a Bayesian approach to the optimization of expensive-to-evaluate functions has already been discussed extensively (especially in Jones [2001]) and confirmed by many applications to industrial design problems (see, e.g., Huang and Allen [2005], Villemonteix et al [2008a]). The context of restricted evaluation budget indeed makes most algorithms inefficient.…”