Cost-aware Multi-objective Bayesian optimisation

Abstract

The notion of "expense" in Bayesian optimisation generally refers to the uniformly expensive cost of function evaluations over the whole search space. However, in some scenarios, the cost of evaluation for black-box objective functions is non-uniform since different inputs from search space may incur different costs for function evaluations. We introduce a cost-aware multi-objective Bayesian optimisation with nonuniform evaluation cost over objective functions by defining cost-aware constraints over the search space. The costaware constraints are a sorted tuple of indexes that demonstrate the ordering of dimensions of the search space based on the user's prior knowledge about their cost of usage. We formulate a new multi-objective Bayesian optimisation acquisition function with detailed analysis of the convergence that incorporates this cost-aware constraints while optimising the objective functions. We demonstrate our algorithm based on synthetic and real-world problems in hyperparameter tuning of neural networks and random forests.

Bayesian optimization for modular black-box systems with switching costs

“…We compared LaMBO with common baselines GP-UCB [14], GP-EI [15], Max-value entropy search [16], random sampling, and three cost-aware strategies: EIpu [17], CA-MOBO [19], and CArBO [18].…”

“…To empirically evaluate the performance of the proposed method, we applied LaMBO to a number of synthetic datasets used in the literature. When compared with traditional Bayesian optimization (BO) baselines [14,15,16] and cost-aware variants [17,18,19], we find that our method outperforms these methods in terms of the trade-off between deviation from global optimum and the cumulative cost. We next apply LaMBO to a problem arising in neuroscience where the goal is to produce a 3D segmentation of brain structure from high-resolution imaging data [20,10].…”

“…Instead of trying to minimize a function purely using the fewest samples, cost-aware methods strive to find the optimizer with least cumulative cost. Common strategies taking the cost explicitly into account in the design of the acquisition function [17,19,18]. Other approaches use dynamic programming to solve this problem with cumulative budget constraints [23,24].…”

A Nonmyopic Approach to Cost-Constrained Bayesian Optimization

“…Swersky et al [2013] introduce a cost-constrained, multi-task variant of entropy search to speed-up optimization of logistic regression and latent Dirichlet allocation. Cost information is input as a set of cost preferences (e.g., a parameter x 1 is more expensive than a parameter x 2 ) by Abdolshah et al [2019], who develop a multi-objective, constrained BO method that evaluates cheap points before expensive ones, as determined by the cost preferences, to find feasible, low-cost solutions. These methods outperform their black-box counterparts by evaluating cheap proxies or cheap points before selecting expensive evaluations, which is accomplished by leveraging additional cost information inside the optimization routine.…”

“…However, we can extend this to a vectorvalued constraint. For example, in materials design, there might be a finite amount of each constituent component, each with its own budget [Abdolshah et al, 2019].…”