Generalized Nonlinear Least Squares Method for the Calibration of Complex Computer Code Using a Gaussian Process Surrogate

Abstract

The approximated nonlinear least squares (ALS) method has been used for the estimation of unknown parameters in the complex computer code which is very time-consuming to execute. The ALS calibrates or tunes the computer code by minimizing the squared difference between real observations and computer output using a surrogate such as a Gaussian process model. When the differences (residuals) are correlated or heteroscedastic, the ALS may result in a distorted code tuning with a large variance of estimation. Another potential drawback of the ALS is that it does not take into account the uncertainty in the approximation of the computer model by a surrogate. To address these problems, we propose a generalized ALS (GALS) by constructing the covariance matrix of residuals. The inverse of the covariance matrix is multiplied to the residuals, and it is minimized with respect to the tuning parameters. In addition, we consider an iterative version for the GALS, which is called as the max-minG algorithm. In this algorithm, the parameters are re-estimated and updated by the maximum likelihood estimation and the GALS, by using both computer and experimental data repeatedly until convergence. Moreover, the iteratively re-weighted ALS method (IRWALS) was considered for a comparison purpose. Five test functions in different conditions are examined for a comparative analysis of the four methods. Based on the test function study, we find that both the bias and variance of estimates obtained from the proposed methods (the GALS and the max-minG) are smaller than those from the ALS and the IRWALS methods. Especially, the max-minG works better than others including the GALS for the relatively complex test functions. Lastly, an application to a nuclear fusion simulator is illustrated and it is shown that the abnormal pattern of residuals in the ALS can be resolved by the proposed methods.

Expectation-Maximization Algorithm for the Calibration of Complex Simulator Using a Gaussian Process Emulator

“…Further details including technical specifications are provided in the Appendix. Certain contents of this study are inevitably similar to those of Reference [ 4 , 6 ] because the problem setting of these studies was very similar.…”

“…If the simulator mimics the real experimental data effectively well without discrepancy, we use the following model to approximate : where e is independent and normally distributed random variable with mean zero and variance . Certain contents of this Appendix are similar to those in References [ 4 , 6 ].…”

“…Our work focuses on calibration in a frequentist manner [ 6 , 7 , 8 ], rather than a Bayesian approach [ 2 , 9 , 10 ]. A Bayesian calibration with a bias correction was introduced by Kennedy and O’Hagan [ 2 ].…”

Artificial Intelligence and Computational Methods in the Modeling of Complex Systems

“…A generalized approximated nonlinear least-squares method by constructing the covariance matrix of residuals is proposed in [ 13 ] by Lee et al The inverse of the covariance matrix is multiplied to the residuals, and it is minimized with respect to the tuning parameters. In addition, an iterative version for the generalized approximated non-linear least-squares namely the max-min G algorithm is considered.…”