Constrained robust Bayesian optimization of expensive noisy black‐box functions with guaranteed regret bounds

Abstract

Many real‐world design problems involve optimization of expensive black‐box functions. Bayesian optimization (BO) is a promising approach for solving such challenging problems using probabilistic surrogate models to systematically tradeoff between exploitation and exploration of the design space. Although BO is often applied to unconstrained problems, it has recently been extended to the constrained setting. Current constrained BO methods, however, cannot identify solutions that are robust to unavoidable uncertainties. In this article, we propose a robust constrained BO method, constrained adversarially robust Bayesian optimization (CARBO), that addresses this challenge by jointly modeling the effect of the design variables and uncertainties on the unknown functions. Using exact penalty functions, we establish a bound on the number of CARBO iterations required to find a near‐global robust solution and provide a rigorous proof of convergence. The advantages of CARBO are demonstrated on two case studies including a non‐convex benchmark problem and a realistic bubble column reactor design problem.

Outlook: How I Learned to Love Machine Learning (A Personal Perspective on Machine Learning in Process Systems Engineering)

“…I also think that the PSE community is uniquely positioned to make contributions on embedding safety and constraint satisfaction logic in learning and decision-making algorithms, as these are critical aspects that arise in the design and operation of chemical processes. 92,93 A key role that the PSE community has played over the years is that of generating problem instances and applications that colleagues in other fields have used to advance math/computing methodologies and tools. For instance, the PSE community has generated countless models that are used to benchmark mixedinteger programming solvers.…”

“…Along these lines, I think that the PSE community can contribute in finding modeling abstractions that embed physics and expert knowledge in different forms (e.g., priors, constraints, logic, reference models) and in quantifying the effect that different types of information have on model generalizability/uncertainty. I also think that the PSE community is uniquely positioned to make contributions on embedding safety and constraint satisfaction logic in learning and decision-making algorithms, as these are critical aspects that arise in the design and operation of chemical processes. , …”

CAPBO: A cost‐aware parallelized Bayesian optimization method for chemical reaction optimization

“…Notably, remarkable theoretical and algorithmic progress has opened the door to the use of for Bayesian optimization to optimize chemical reactions. [22][23][24][25][26][27][28][29][30] For example, Häse et al incorporated categorical variables and expert knowledge to construct the Gryffin optimizer, 23 while Hickman et al considered and tackled constrained optimization problems. 24 Some open-source libraries and software involving functional integration and GPU acceleration have also been developed in recent years.…”

“…Bayesian optimization has also been combined with derivative‐free optimization algorithms such as the Nelder–Mead simplex method 20 and neural networks 21 to achieve better performance. Notably, remarkable theoretical and algorithmic progress has opened the door to the use of for Bayesian optimization to optimize chemical reactions 22–30 . For example, Häse et al incorporated categorical variables and expert knowledge to construct the Gryffin optimizer, 23 while Hickman et al considered and tackled constrained optimization problems 24 .…”