A comparison of Gaussian processes and neural networks for computer model emulation and calibration

Abstract

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well-calibrated uncertainty estimates.The neural network emulator is accurate, but tends to have large uncertainty on its predictions. As a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

Optimization of Mixed Micelles Based on Oppositely Charged Block Copolymers by Machine Learning for Application in Gene Delivery

“…Such a model works with minimal data and can be improved iteratively by adding new data points. [ 59 ] The resulting model can predict an expected outcome at every data point in the parameter space, usually in a smooth hyperplane. Furthermore, the model come with a standard deviation that can be evaluated at every point in the input parameter space, giving a metric for the uncertainty of the model, which can be used for the exploration of formulations where the model has only very little knowledge.…”

Active learning for efficient navigation of multi-component gas adsorption landscapes in a MOF

“…[48][49][50] Furthermore, GPs have shown better performance for small sized datasets compared with models like Bayesian neural networks (BNNs) or neural network ensembles, which can also provide uncertainty estimates. 51,52 This makes GPs well suited for predictions for adsorption like problems where generating large dataset is expensive. These approaches are already gaining popularity in the molecular simulation space.…”

Hyperparameter Bayesian Optimization of Gaussian Process Regression Applied in Speed-Sensorless Predictive Torque Control of an Autonomous Wind Energy Conversion System

“…In hyperparameter optimization utilizing the BO algorithm, in the beginning, it is necessary to define an initial set of configurations, X 0 (input value determined using the gray box principle described in the next section), and their associated function (in our case, the machine flux and speed value): D 0 = {X 0 , y 0 }. Then, in a loop, we updated the GP model using the Bayes rule; see [53,54]. Subsequently, a new hyperparameter configuration was chosen through the optimization of an auxiliary function known as the acquisition function.…”