An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization

Abstract

Response surface methods based on kriging and radial basis function (RBF) interpolation have been successfully applied to solve expensive, i.e. computationally costly, global black-box nonconvex optimization problems. In this paper we describe extensions of these methods to handle linear, nonlinear, and integer constraints. In particular, algorithms for standard RBF and the new adaptive RBF (ARBF) are described. Note, however, while the objective function may be expensive, we assume that any nonlinear constraints are either inexpensive or are incorporated into the objective function via penalty terms. Test results are presented on standard test problems, both nonconvex problems with linear and nonlinear constraints, and mixed-integer nonlinear problems (MINLP). Solvers in the TOMLAB Optimization Environment (http://tomopt.com/tomlab/) have been compared, specifically the three deterministic derivative-free solvers rbfSolve, ARBFMIP and EGO with three derivative-based mixed-integer nonlinear solvers, OQNLP, MINLPBB and MISQP, as well as the GENO solver implementing a stochastic genetic algorithm. Results show that the deterministic derivative-free methods compare well with the derivativebased ones, but the stochastic genetic algorithm solver is several orders of magnitude too slow for practical use. When the objective function for the test problems is costly to evaluate, the performance of the ARBF algorithm proves to be superior.

Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO

“…Other global-search surrogate-based methods which employ kriging approximations within box constrained regions have been developed by Forrester and Jones (2008) and Quan et al (2013). Radial-basis functions have also been used in the global-search surrogate-based CDFO methods for box-constrained problems (Björkman and Holmström, 2000;Holmström et al, 2008b;Jakobsson et al, 2010;Shoemaker, 2007a,b,c, 2013b). Yao et al (2014) use a hybrid Neural-Network Radial-basis function model within a global-search framework which forces gradient estimations in local regions to agree with actual gradient information.…”

“…Villemonteix et al (2009a,b) introduce a new minimizer entropy kriging-based criterion for locating new samples, which must be evaluated using conditional simulations in order to incorporate constraint satisfaction, while Kleijnen et al (2010) develop a method for solving constrained Integer Nonlinear Problems using kriging approximations. General constraints have been approximated by radial basis functions in the works of Regis and Shoemaker (2005); Regis (2011Regis ( , 2014; Rashid et al (2013), while Holmström et al (2008b) incorporate constraints in an aggregated penalty term which is added to the objective function, transforming the problem to box-constrained. Ierapetritou (2013, 2014) employ kriging approximations for the objective function and the lumped feasibility-based criterion which aims to represent the feasible space of the problem through a single function.…”

On the safety of Gomory cut generators

“…Black-box optimization methods are typically tailored for continuous problems, avoiding the difficulty of dealing with discrete variables. More recently, some attempts at solving problems with integer variables have been made [17,18].…”

A surrogate model assisted evolutionary algorithm for computationally expensive design optimization problems with discrete variables

“…For SBNOAs, successful methods that handle discrete numerical variables include [5], [8], [9], etc. The generation of possible promising candidate solutions is based on a random sampling strategy [5], solving an auxiliary optimization problem [9], or traditional heuristic search methods [8] such as branch and bound.…”

“…The generation of possible promising candidate solutions is based on a random sampling strategy [5], solving an auxiliary optimization problem [9], or traditional heuristic search methods [8] such as branch and bound. A reasonably optimal solution can often be obtained within a limited number of exact evaluations.…”