Investigating Bayesian optimization for expensive-to-evaluate black box functions: Application in fluid dynamics

Abstract

Bayesian optimization (BO) provides an effective method to optimize expensive-to-evaluate black box functions. It has been widely applied to problems in many fields, including notably in computer science, e.g., in machine learning to optimize hyperparameters of neural networks, and in engineering, e.g., in fluid dynamics to optimize control strategies that maximize drag reduction. This paper empirically studies and compares the performance and the robustness of common BO algorithms on a range of synthetic test functions to provide general guidance on the design of BO algorithms for specific problems. It investigates the choice of acquisition function, the effect of different numbers of training samples, the exact and Monte Carlo (MC) based calculation of acquisition functions, and both single-point and multi-point optimization. The test functions considered cover a wide selection of challenges and therefore serve as an ideal test bed to understand the performance of BO to specific challenges, and in general. To illustrate how these findings can be used to inform a Bayesian optimization setup tailored to a specific problem, two simulations in the area of computational fluid dynamics (CFD) are optimized, giving evidence that suitable solutions can be found in a small number of evaluations of the objective function for complex, real problems. The results of our investigation can similarly be applied to other areas, such as machine learning and physical experiments, where objective functions are expensive to evaluate and their mathematical expressions are unknown.

Optimisation and Analysis of Streamwise-Varying Wall-Normal Blowing in a Turbulent Boundary Layer

“…The signal variance and the length-scales of the kernel are estimated via maximum likelihood estimation. The acquisition function for this work is chosen to be UCB, with a trade-off parameter = 1 , as it was found to perform well on a variety of different test problems (Diessner et al 2022). The parallel strategy briefly described in Sect.…”

“…This suggests that in all 30 Bayesian optimisation runs a close-to-optimal solution is found. For more details on the Bayesian optimisation framework and its validation, the reader is referred to Diessner et al (2022).…”

“…Therefore, both studies share some similarities, such as the same flow problem, solver and power estimation method. However, the key novelties of this work are: (1) investigation of a novel blowing strategy; (2) the validation and use of implicit LES data, instead of DNS data, to extract the NES values; (3) a detailed analysis of the flow physics for selected cases, including a comparison between effective and ineffective configurations; (4) the use of a state-of-the-art in-house Bayesian optimisation framework (Diessner et al 2022). In the following sections the case setup is introduced, followed by a description of the flow solver and Bayesian optimisation framework.…”

Low Frequency Tibial Neuromodulation Increases Voiding Activity - A Computational Model

“…Bayesian optimisation was used to fit the parameters of the model 29 . To do so, the activity of second order sensory afferents was extracted, downsampled to 10Hz, and smoothed (1st order Butterworth Filter, 1Hz cutoff frequency).…”

Bayesian optimization of single-atom alloys and other bimetallics: efficient screening for alkane transformations, CO2 reduction, and hydrogen evolution

“…This makes BO ideal in the evaluation of expensive objective functions. BO has excelled in many domains [22][23][24] and is one of the underlying technologies behind most self-driving labs. [25][26][27] One major advantage of BO over traditional ML algorithms for catalyst discovery is its capacity to handle multiple objectives and constraints over the search space.…”