Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems

Abstract

Gaussian processes (GPs) are the most popular model used in surrogate-assisted evolutionary optimization of computationally expensive problems, mainly because GPs are able to measure the uncertainty of the estimated fitness values, based on which certain infill sampling criteria can be used to guide the search and update the surrogate model. However, the computation time for constructing GPs may become excessively long when the number of training samples increases, which makes it inappropriate to use them as surrogates in evolutionary optimization. To address this issue, this paper proposes to use ensembles as surrogates and infill criteria for model management in evolutionary optimization. A heterogeneous ensemble consisting of a least square support vector machine and two radial basis function networks is constructed to enhance the reliability of ensembles for uncertainty estimation. In addition to the original decision variables, a selected subset of the decision variables and a set of transformed variables are used as inputs of the heterogeneous ensemble to further promote the diversity of the ensemble. The proposed heterogeneous ensemble is compared with a GP and a homogeneous ensemble for infill sampling criteria in evolutionary multiobjective optimization. Experimental results demonstrate that the heterogeneous ensemble is competitive in performance compared with GPs and much more scalable in computational complexity to the increase in search dimension.

Data-Driven Evolutionary Optimization: An Overview and Case Studies

“…In addition, the distance from the sample solution to the existing training data has been used as an uncertainty measure in [46]. Finally, ensemble machine learning models have been proved to be promising in providing the uncertainty information, where the variance of the predictions outputted by the base learners of the ensemble can be used to estimate the degree of uncertainty in fitness prediction [49], [50]. Both promising and uncertain samples are important for online data-driven EAs.…”

“…Generally speaking, however, stochastic models such as Kriging models may be preferred if an infill criterion is to be used for model management. As discussed in [49], the main limitation of Kriging models is their possibly large computational complexity when a large number of training samples is involved. In this case, ensembles are good alternatives to Kriging models due to their scalable computational complexity.…”

Model-based evolutionary algorithms: a short survey

“…Since Kriging models are able to provide uncertainty information in the form of a confidence level of the predicted fitness [48], they have recently become very popular in SAEAs. To take advantage of the uncertainty information provided by the Kriging models, various model management criteria, also known as infill sampling criteria in the Kriging-assisted optimization, have been proposed in SAEAs, such as the probability of improvement (PoI) [65,66], the expected improvement (ExI) [67], the lower confidence bound (LCB) [64], and the heterogeneous ensemble-based infill criterion to enhance the reliability of ensembles for uncertainty estimation [68].…”

Offline Data-Driven Evolutionary Optimization Using Selective Surrogate Ensembles

“…Infill criteria, including expected improvement [32], lower confidence bound (LCB) [33], [34], and probability of improvement [35], are most widely used in Kriging or Gaussian process assisted EAs. Most recently, infill criteria have been extended to surrogates consisting of heterogeneous ensembles [36].…”