Efficient optimization of the likelihood function in Gaussian process modelling

Abstract

Gaussian Process (GP) models are popular statistical surrogates used for emulating computationally expensive computer simulators. The quality of a GP model fit can be assessed by a goodness of fit measure based on optimized likelihood. Finding the global maximum of the likelihood function for a GP model is typically very challenging as the likelihood surface often has multiple local optima, and an explicit expression for the gradient of the likelihood function is typically unavailable. Previous methods for optimizing the likelihood function (e.g., MacDonald et al. (2013)) have proven to be robust and accurate, though relatively inefficient. We propose several likelihood optimization techniques, including two modified multi-start local search techniques, based on the method implemented by MacDonald et al. (2013), that are equally as reliable, and significantly more efficient. A hybridization of the global search algorithm Dividing Rectangles (DIRECT) with the local optimization algorithm BFGS provides a comparable GP model quality for a fraction of the computational cost, and is the preferred optimization technique when computational resources are limited. We use several test functions and a real application from an oil reservoir simulation to test and compare the performance of the proposed methods with the one implemented by MacDonald et al. (2013) in the R library GPfit. The proposed method is implemented in a Matlab package, GPMfit.

Inverse Problem for a Time-Series Valued Computer Simulator via Scalarization

“…For instance, we should use a variety of test functions, repeat the simulations to average out the effect of initial design choice, find optimal (n 0 , n new ) combination in the sequential procedure, and so on. (2) The scalarization process should be further strengthened by using more informative discrepancy measure as compared to Euclidean distance. (3) Does this scalarization procedure affect the likeliness of finding the inverse solution?…”

“…See [10] for more details. Popular implementations of the GP model like mlegp [11], GPfit ( [12]), GPmfit [13], and DiceKriging ( [14]) use some sort of numerical fix to overcome the computational instability issue. We used GPfit in R for all implementations of the GP model.…”

“…It turns out that the EI surfaces are typically multi-modal, and the location and number of prominent modes/peaks changes from iteration-to-iteration. Popular optimization techniques used for EI optimization include, genetic algorithm ([1]), particle swarm optimization [13], multistart newton-based methods ( [12]), and branch-and-bound algorithms [17]. Thus, one should be careful in choosing the follow-up points to achieve the optimal improvement at every step.…”

Statistical Modelling and Analysis of the Computer-Simulated Datasets

“…MRST model: Finding the optimal drilling locations for production and injection wells in an oil reservoir is of utmost importance (see Onwunalu and Durlofsky, 2010). Butler et al (2014) used a Matlab Reservoir Simulator (MRST) (Lie et al, 2012) to generate the anticipated net present value (NPV) of the produced oil for a well to be drilled at a particular location. The goal here was to determine the configuration of wells that yields the best NPV.…”

The Problem with Assessing Statistical Methods

“…Chen et al (2015) discuss the importance of having extensive simulations and multiple data sets to validate a new statistical procedure. This working example and an incomplete list of other papers (Loeppky et al, 2009(Loeppky et al, , 2010Williams et al, 2011;Gramacy and Lee, 2012;Gramacy and Apley, 2014;Butler et al, 2014;Lam and Notz, 2008) all have a similar structure of a reasonably comprehensive set of simulations and number of methods that need to be assessed. The papers above typically show the results as a series of sideby-side boxplots or side-by-side dotcharts for each method, with one plot for each test function and sample size.…”