Evolution Strategies with Additive Noise

Astete-Morales, Sandra; Cauwet, Marie-Liesse; Teytaud, Olivier

doi:10.1145/2725494.2725500

Cited by 17 publications

(15 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This rate s(SR) = −1 has been proved tight in [14]. Hence, Shamir and Fabian algorithm are faster than ES's, which cannot do better than s(SR) = − 1 2 , at least under their usual form [4].…”

Section: Stochastic Gradient Descentmentioning

confidence: 94%

“…The convergence rate is s(SR) = K for some K < 0 under assumptions about the convergence in ES in the noise-free case. Moreover [4] shows that ES, under general conditions, must exhibit K > − 1 2 . There is no formal proof of an upper bound that can theoretically ensure a value or a range for s(SR).…”

Section: Simple Regret For Es With Resamplingsmentioning

confidence: 95%

“…The name is proposed by us. 4 If (k − ⌊g(k)⌋) < 0, then the max on the definition considers indexes between 1 and k explained by the following argument: g(k) be large enough, so that we have a correct recommendation confirmed over g(k) iterations, but small enough, so that we do not have to wait many evaluations before acknowledging that a correct recommendation has been found. The RSR uses the recommendatioñ x m instead of the search point x m used in ASR.…”

Section: Simple Regret and Variantsmentioning

confidence: 99%

See 2 more Smart Citations

Analysis of Different Types of Regret in Continuous Noisy Optimization

Astete-Morales

Cauwet

Teytaud

2016

Proceedings of the Genetic and Evolutionary Computation Conference 2016

Self Cite

View full text Add to dashboard Cite

The performance measure of an algorithm is a crucial part of its analysis. The performance can be determined by the study on the convergence rate of the algorithm in question. It is necessary to study some (hopefully convergent) sequence that will measure how "good" is the approximated optimum compared to the real optimum.The concept of Regret is widely used in the bandit literature for assessing the performance of an algorithm. The same concept is also used in the framework of optimization algorithms, sometimes under other names or without a specific name. And the numerical evaluation of convergence rate of noisy algorithms often involves approximations of regrets. We discuss here two types of approximations of Simple Regret used in practice for the evaluation of algorithms for noisy optimization. We use specific algorithms of different nature and the noisy sphere function to show the following results. The approximation of Simple Regret, termed here Approximate Simple Regret, used in some optimization testbeds, fails to estimate the Simple Regret convergence rate. We also discuss a recent new approximation of Simple Regret, that we term Robust Simple Regret, and show its advantages and disadvantages.

show abstract

Section: Stochastic Gradient Descentmentioning

confidence: 94%

Section: Simple Regret For Es With Resamplingsmentioning

confidence: 95%

Section: Simple Regret and Variantsmentioning

confidence: 99%

See 1 more Smart Citation

Analysis of Different Types of Regret in Continuous Noisy Optimization

Astete-Morales

Cauwet

Teytaud

2016

Proceedings of the Genetic and Evolutionary Computation Conference 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is difficult to make ES noise-resilient. The lower bound of the convergence-rate in terms of total number of function evaluations T has been shown to be Ω 1/ √ T [2]. The lower bound relies on the property that ESs adapt the step-size to be O (∥x − x * ∥/d).…”

Section: Introductionmentioning

confidence: 99%

“…The lower bound relies on the property that ESs adapt the step-size to be O (∥x − x * ∥/d). It has been shown that ensuring a larger step-size allows for O(1/T ) convergence on a quadratic function, the theoretical optimum [2,8].…”

Section: Introductionmentioning

confidence: 99%

Large-scale noise-resilient evolution-strategies

Krause

2019

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Ranking-based Evolution Strategies (ES) are efficient algorithms for problems where gradient-information is not available or when the gradient is not informative. This makes ES interesting for Reinforcement-Learning (RL). However, in RL the high dimensionality of the search-space, as well as the noise of the simulations make direct adaptation of ES challenging. Noise makes ranking points difficult and a large budget of re-evaluations is needed to maintain a bounded error rate. In this work, the ranked weighting is replaced by a linear weighting function, which results in nearly unbiased stochastic gradient descent (SGD) on the manifold of probability distributions. The approach is theoretically analysed and the algorithm is adapted based on the results of the analysis. It is shown that in the limit of infinite dimensions, the algorithm becomes invariant to smooth monotonous transformations of the objective function. Further, drawing on the theory of SGD, an adaptation of the learning-rates based on the noise-level is proposed at the cost of a second evaluation for every sampled point. It is shown empirically that the proposed method improves on simple ES using Cumulative Step-size Adaptation and ranking. Further, it is shown that the proposed algorithm is more noise-resilient than a ranking-based approach.

show abstract

Evolution Under Strong Noise: A Self-Adaptive Evolution Strategy Can Reach the Lower Performance Bound - The pcCMSA-ES

Hellwig

Beyer

2016

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Evolution Strategies with Additive Noise

Cited by 17 publications

References 23 publications

Analysis of Different Types of Regret in Continuous Noisy Optimization

Analysis of Different Types of Regret in Continuous Noisy Optimization

Large-scale noise-resilient evolution-strategies

Evolution Under Strong Noise: A Self-Adaptive Evolution Strategy Can Reach the Lower Performance Bound - The pcCMSA-ES

Contact Info

Product

Resources

About