Comparison-Based Optimizers Need Comparison-Based Surrogates

Loshchilov, Ilya; Schoenauer, Marc; Sebag, Michèle

doi:10.1007/978-3-642-15844-5_37

Cited by 79 publications

(57 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Castelletti et al (2010) developed a multi-objective optimization method for water quality management using radial basis function (RBF), inverse distance weighted and n-dimensional linear interpolator as surrogates. Loshchilov et al (2010) investigated the use of the ranked-based support vector machine (SVM) and demonstrated that for surrogate-based optimization capturing the relative value of the objective functions is more important than reducing the absolute fitting error. Pilát and Neruda (2013) developed a surrogate model selector for multi-objective surrogate-assisted optimization.…”

Section: Introductionmentioning

confidence: 99%

Multi-objective parameter optimization of common land model using adaptive surrogate modeling

Gong

Duan

et al. 2015

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract. Parameter specification usually has significant influence on the performance of land surface models (LSMs). However, estimating the parameters properly is a challenging task due to the following reasons: (1) LSMs usually have too many adjustable parameters (20 to 100 or even more), leading to the curse of dimensionality in the parameter input space; (2) LSMs usually have many output variables involving water/energy/carbon cycles, so that calibrating LSMs is actually a multi-objective optimization problem; (3) Regional LSMs are expensive to run, while conventional multi-objective optimization methods need a large number of model runs (typically ∼ 10 5 -10 6 ). It makes parameter optimization computationally prohibitive. An uncertainty quantification framework was developed to meet the aforementioned challenges, which include the following steps: (1) using parameter screening to reduce the number of adjustable parameters, (2) using surrogate models to emulate the responses of dynamic models to the variation of adjustable parameters, (3) using an adaptive strategy to improve the efficiency of surrogate modeling-based optimization; (4) using a weighting function to transfer multi-objective optimization to single-objective optimization. In this study, we demonstrate the uncertainty quantification framework on a single column application of a LSM -the Common Land Model (CoLM), and evaluate the effectiveness and efficiency of the proposed framework. The result indicate that this framework can efficiently achieve optimal parameters in a more effective way. Moreover, this result implies the possibility of calibrating other large complex dynamic models, such as regionalscale LSMs, atmospheric models and climate models.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-objective parameter optimization of common land model using adaptive surrogate modeling

Gong

Duan

et al. 2015

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

show abstract

“…the objective function is modeled as a convexquadratic function and thus the algorithm exploits that some functions can be locally approximated by quadratic models. Consequently, invariance to rank-preserving transformations is lost [2,13] explaining on the other hand the improvements that are observed on functions that can be locally accurately modeled as quadratic functions.…”

Section: Introductionmentioning

confidence: 99%

“…With the motivation to preserve the invariance to monotonic transformations, rank-based Support Vector Machine were introduced as surrogate models within CMA-ES in the ACM-ES algorithm [13]. Speedups with respect to CMA-ES between 0.5 and 4.5 were observed on seven uni-and multimodal benchmark functions, with dimension in-between 2 and 40.…”

Section: Introductionmentioning

confidence: 99%

Local-meta-model CMA-ES for partially separable functions

Bouzarkouna

Auger

Ding

2011

Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation

View full text Add to dashboard Cite

In this paper, we propose a new variant of the covariance matrix adaptation evolution strategy with local meta-models (lmm-CMA) for optimizing partially separable functions. We propose to exploit partial separability by building at each iteration a meta-model for each element function (or sub-function) using a full quadratic local model. After introducing the approach we present some first experiments using element functions with dimensions 2 and 4. Our results demonstrate that, as expected, exploiting partial separability leads to an important speedup compared to the standard CMA-ES. We show on the tested functions that the speedup increases with increasing dimensions for a fixed dimension of the element function. On the standard Rosenbrock function the maximum speedup of λ is reached in dimension 40 using element functions of dimension 2. We show also that higher speedups can be achieved by increasing the population size. The choice of the number of points used to build the meta-model is also described and the computational cost is discussed.

show abstract

“…Furthermore, in order to preserve the invariance properties of the ACiD, comparison-based surrogate models can be used, as advocated in [9].…”

Section: Discussionmentioning

confidence: 99%

Adaptive coordinate descent

Loshchilov

Schoenauer

Sebag

2011

Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

Independence from the coordinate system is one source of efficiency and robustness for the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). The recently proposed Adaptive Encoding (AE) procedure generalizes CMA-ES adaptive mechanism, and can be used together with any optimization algorithm. Adaptive Encoding gradually builds a transformation of the coordinate system such that the new coordinates are as decorrelated as possible with respect to the objective function. But any optimization algorithm can then be used together with Adaptive Encoding, and this paper proposes to use one of the simplest of all, that uses a dichotomy procedure on each coordinate in turn. The resulting algorithm, termed Adaptive Coordinate Descent (ACiD), is analyzed on the Sphere function, and experimentally validated on BBOB testbench where it is shown to outperform the standard (1 + 1)-CMA-ES, and is found comparable to other state-of-the-art CMA-ES variants.

show abstract

Comparison-Based Optimizers Need Comparison-Based Surrogates

Cited by 79 publications

References 13 publications

Multi-objective parameter optimization of common land model using adaptive surrogate modeling

Multi-objective parameter optimization of common land model using adaptive surrogate modeling

Local-meta-model CMA-ES for partially separable functions

Adaptive coordinate descent

Contact Info

Product

Resources

About