Bayesian Optimisation for Mixed-Variable Inputs using Value Proposals

Abstract

Many real-world optimisation problems are defined over both categorical and continuous variables, yet efficient optimisation methods such as Bayesian Optimisation (BO) are not designed to handle such mixedvariable search spaces. Recent approaches to this problem cast the selection of the categorical variables as a bandit problem, operating independently alongside a BO component which optimises the continuous variables. In this paper, we adopt a holistic view and aim to consolidate optimisation of the categorical and continuous sub-spaces under a single acquisition metric. We derive candidates from the Expected Improvement criterion, which we call value proposals, and use these proposals to make selections on both the categorical and continuous components of the input. We show that this unified approach significantly outperforms existing mixed-variable optimisation approaches across several mixed-variable black-box optimisation tasks.

Mixed-Variable Black-Box Optimisation Using Value Proposal Trees

“…The GP models an underlying probability distribution over functions f and is represented by mean and covariance functions (or kernel) µ(x) and κ(x, x ′ ) respectively, where f (x) ∼ GP(µ(x), κ(x, x ′ )). In this work, our choice of kernel κ is the mixed-kernel (Ru et al 2020;Parker-Holder et al 2021;Wan et al 2021;Zuo et al 2022) which is suited towards modelling mixed-input type problems (for additional details on the mixed-kernel, please refer to the supplementary material).…”