Improved Gaussian Process Acquisition for Targeted Bayesian Optimization

Abstract

A black-box optimization problem is considered, in which the function to be optimized can only be expressed in terms of a complicated stochastic algorithm that takes a long time to evaluate. The value returned is required to be sufficiently near to a target value, and uses data that has a significant noise component. Bayesian Optimization with an underlying Gaussian Process is used as an optimization solution, and its effectiveness is measured in terms of the number of function evaluations required to attain the target. To improve results, a simple modification of the Gaussian Process ‘Lower Confidence Bound’ (LCB) acquisition function is proposed. The expression used for the confidence bound is squared in order to better comply with the target requirement. With this modification, much improved results compared to random selection methods and to other commonly used acquisition functions are obtained.

Operational Risk Reverse Stress Testing: Optimal Solutions

“…We have published extensive results for zero acquisition, including comparisons with "established" acquisition functions. They may be found in [35,36]. Those results are too extensive to reproduce in full in this paper, but we do report a summary of them below.…”

“…The main findings from [35,36] were (the list below shows the means of run numbers with standard deviations in square brackets): A qualitative summary of previous results is that Block 1 ("traditional") methods have the same performance characteristics as random selection, and that zero acquisitions approximately halves both the "traditional" means and standard deviations run number. In the sections that follow, our current results are compared to our previous results.…”

“…Zero acquisition was discussed in detail in [35,36]. The intuition behind the proposal in Equation ( 5) is that since g has to be as close as possible to a target, a simple way to measure closeness is "deviation-squared" (absolute deviation works just as well).…”

Probabilistic surrogate modeling by Gaussian process: A review on recent insights in estimation and validation

“…On the other side, the availability of gradients in the MLE case (without significant additional computational cost) is an advantage in the implementation of numerical optimization algorithms required for hyperparameter estimation. However, this advantage must be qualified by the recent works on the computational complexity of cross-validation schemes and more precisely the fast computation of gradients of LOO criteria [58,43,50]. Still in Petit [50], an intensive benchmark on analytic functions of different dimensions shows that MLE is often preferable to its competitors (not only in well specified cases but also in case of overestimated regularity), and that the choice of regularity (ν in Matérn class) might be often more important than the estimation of GP hyperparameters.…”

“…Hence, as suggested by Petit [50] and in the direct line of Demay et al [44], an interesting compromise (that we also recommend from our experience) is to consider a finite collection of covariance functions (those of Table 1), then estimate the hyperparameters (σ 2 , θ) for each of them, and finally use a validation criterion (different from criterion used for the estimation) to select the best covariance.…”