Kriging Is Well-Suited to Parallelize Optimization

Infill sampling criteria for surrogate-based optimization with constraint handling

“…With the availability of parallel computing becoming commonplace, formulation of multiple update infill criteria has received further attention in recent years (Henkenjohann and Kunert 2007, Ponweiser et al 2008, Viana and Haftka 2010b. Most recently, Ginsbourger et al (2010) have provided an extension to Schonlau's early work by providing a sound analytical expression for finding multiple updates based on the expected improvement. This expression becomes mathematically intractable beyond two updates at each iteration and relies on expensive statistical methods such as Monte Carlo simulations for a higher number of multiple updates.…”

Fast Computation of the Multi-Points Expected Improvement with Applications in Batch Selection

“…Coming back to the decision-theoretic roots of EI [5], a Multi-points Expected Improvement (also called "q-EI") criterion for batch-sequential optimization was defined in [6] and further developed in [7,8]. Maximizing this criterion enables choosing batches of q > 1 points at which to evaluate f in parallel, and is of particular interest in the frequent case where several CPUs are simultaneously available.…”

“…Maximizing this criterion enables choosing batches of q > 1 points at which to evaluate f in parallel, and is of particular interest in the frequent case where several CPUs are simultaneously available. Even though an analytical formula was derived for the 2-EI in [7], the Monte Carlo (MC) approach of [8] for computing q-EI when q ≥ 3 makes the criterion itself expensive-to-evaluate, and particularly hard to optimize.…”

“…The pragmatic approach proposed by [8] consists in circumventing a direct q-EI maximization, and replacing it by simpler strategies where batches are obtained using an offline q-points EGO. In such strategies, the model updates are done using dummy response values such as the kriging mean prediction (Kriging Believer) or a constant (Constant Liar), and the covariance parameters are re-estimated only when real data is assimilated.…”

Bayesian Adaptive Reconstruction of Profile Optima and Optimizers

“…Note that in the example above, the two designs of experiments are generated independently of each other. In practice, however, one rather needs to augment an existing design with new well-chosen points, be it in a sequential (one point after the other, like in [18]) or in a batch-sequential manner [14].…”