Comparing neural networks and Kriging for fitness approximation in evolutionary optimization

Willmes, Lars; Bäck, Thomas; Jin, Yaochu; Sendhoff, Bernhard

doi:10.1109/cec.2003.1299639

Cited by 44 publications

(11 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The incorporation of the fitness model leads to a higher probability of premature convergence in local minima. This problem of model assistance approaches on multimodal problems is also observed by other authors [12,44].…”

Section: Comparative Studiessupporting

confidence: 57%

See 1 more Smart Citation

Model Assisted Evolution Strategies

Ulmer

Streichert

Zell

2005

Knowledge Incorporation in Evolutionary Computation

View full text Add to dashboard Cite

Summary. In many engineering optimization problems the number of fitness function evaluations is limited by time and cost. These problems pose a special challenge to the field of evolutionary computation, since existing evolutionary methods require a very large number of problem function evaluations. One innovative way to address this challenge is the application of approximation models as a surrogate of the real fitness function. Thereby two major points have to be considered. The selection of an appropriate model to approximate the fitness landscape and the coupling of model with the evolutionary algorithm. We discuss both points in detail and investigate different alternatives how knowledge from the model can support the evolutionary optimization process. Special attention is given to probabilistic models like Gaussian Processes, which have the advantage of providing a probabilistic interpretation of the model prediction. We describe a model assisted Evolution Strategy, which uses a Gaussian Process approximation model to preselect the most promising solutions. To refine the preselection process we determine the likelihood of each individual to improve the overall best found solution. Due to this, the new algorithm has a much better convergence behavior and achieves better results than standard evolutionary optimization approaches with less fitness evaluations. Numerical results from extensive simulations on several high dimensional test functions including multimodal functions are presented. These results show that the incorporation of knowledge by fitness approximation considerably enhances the performance of Evolution Strategies.

show abstract

Section: Comparative Studiessupporting

confidence: 57%

“…Gaussian Processing [13,42) and Kriging [14,29) are statistical modeling techniques, which are also used for fitness function approximation. A comparison of neural networks and kriging for fitness approximation in evolutionary optimization can be found in (44).…”

Section: Related Workmentioning

confidence: 99%

Model Assisted Evolution Strategies

Ulmer

Streichert

Zell

2005

Knowledge Incorporation in Evolutionary Computation

View full text Add to dashboard Cite

show abstract

“…GPs have been equally popular in surrogate-assisted single [23], [24] and multi-objective evolutionary optimization [25]- [28]. However, most GP-assisted EAs have been tested only on low-dimensional problems (up to 10 decision variables) [15], mainly due to the fact that the computational cost of constructing the GP is O(N 3 ), where N is the number of training data [29].…”

Section: Introductionmentioning

confidence: 99%

Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems

Guo

Jin

Ding

et al. 2019

IEEE Trans. Cybern.

177

View full text Add to dashboard Cite

Gaussian processes (GPs) are the most popular model used in surrogate-assisted evolutionary optimization of computationally expensive problems, mainly because GPs are able to measure the uncertainty of the estimated fitness values, based on which certain infill sampling criteria can be used to guide the search and update the surrogate model. However, the computation time for constructing GPs may become excessively long when the number of training samples increases, which makes it inappropriate to use them as surrogates in evolutionary optimization. To address this issue, this paper proposes to use ensembles as surrogates and infill criteria for model management in evolutionary optimization. A heterogeneous ensemble consisting of a least square support vector machine and two radial basis function networks is constructed to enhance the reliability of ensembles for uncertainty estimation. In addition to the original decision variables, a selected subset of the decision variables and a set of transformed variables are used as inputs of the heterogeneous ensemble to further promote the diversity of the ensemble. The proposed heterogeneous ensemble is compared with a GP and a homogeneous ensemble for infill sampling criteria in evolutionary multiobjective optimization. Experimental results demonstrate that the heterogeneous ensemble is competitive in performance compared with GPs and much more scalable in computational complexity to the increase in search dimension.

show abstract

“…Many machine learning models can be used for surrogates, such as linear, nonlinear or polynomial repression models [31], Kriging or Gaussian processes [32], [33], [34], [35], [36], [37], [38], support vector machines (SVMs) [39], radial basis function (RBF) networks [40], [41], [42], and many other neural networks [43], [44], [45], [46], [47]. Several ideas have been proposed for choosing individuals to be re-evaluated using the original objective functions, which is one key issue in surrogate management.…”

Section: B Surrogate Models and Surrogate Managementmentioning

confidence: 99%

Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System

Wang

Jin

Jansen

2016

IEEE Trans. Evol. Computat.

214

102

View full text Add to dashboard Cite

Abstract-Most existing work on evolutionary optimization assumes that there are analytic functions for evaluating the objectives and constraints. In the real-world, however, the objective or constraint values of many optimization problems can be evaluated solely based on data and solving such optimization problems is often known as data-driven optimization. In this paper, we divide data-driven optimization problems into two categories, i.e., off-line and on-line data-driven optimization, and discuss the main challenges involved therein. An evolutionary algorithm is then presented to optimize the design of a trauma system, which is a typical off-line data-driven multi-objective optimization problem, where the objectives and constraints can be evaluated using incidents only. As each single function evaluation involves large amount of patient data, we develop a multi-fidelity surrogate management strategy to reduce the computation time of the evolutionary optimization. The main idea is to adaptively tune the approximation fidelity by clustering the original data into different numbers of clusters and a regression model is constructed to estimate the required minimum fidelity. Experimental results show that the proposed algorithm is able to save up to 90% of computation time without much sacrifice of the solution quality.

show abstract

Comparing neural networks and Kriging for fitness approximation in evolutionary optimization

Cited by 44 publications

References 10 publications

Model Assisted Evolution Strategies

Model Assisted Evolution Strategies

Heterogeneous Ensemble-Based Infill Criterion for Evolutionary Multiobjective Optimization of Expensive Problems

Data-Driven Surrogate-Assisted Multiobjective Evolutionary Optimization of a Trauma System

Contact Info

Product

Resources

About