Efficient natural evolution strategies

Sun, Yu; Wierstra, Daan; Schaul, Tom; Schmidhuber, Juergen

doi:10.1145/1569901.1569976

Cited by 83 publications

(104 citation statements)

References 7 publications

Supporting

Mentioning

103

Contrasting

Unclassified

Order By: Relevance

“…A more general approach for partially observable environments directly evolves programs for RNN with internal states (no need for the Markovian assumption), by applying evolutionary algorithms [30,50,83] to RNN weight matrices [26,85,89,105]. Recent work brought progress through a focus on reducing search spaces by co-evolving the comparatively small weight vectors of individual neurons and synapses [21], by Natural Gradient-based Stochastic Search Strategies [19,53,92,93,102,103], and by reducing search spaces through weight matrix compression [36,61]. Our RL RNN now outperform many previous methods on benchmarks [21], creating memories of important events and solving numerous tasks unsolvable by classical RL methods.…”

Section: Recurrent / Deep Neural Networkmentioning

confidence: 99%

See 1 more Smart Citation

New Millennium AI and the Convergence of History: Update of 2012

Schmidhuber

2012

The Frontiers Collection

View full text Add to dashboard Cite

Summary. Artificial Intelligence (AI) has recently become a real formal science: the new millennium brought the first mathematically sound, asymptotically optimal, universal problem solvers, providing a new, rigorous foundation for the previously largely heuristic field of General AI and embedded agents. There also has been rapid progress in not quite universal but still rather general and practical artificial recurrent neural networks for learning sequenceprocessing programs, now yielding state-of-the-art results in real world applications. And the computing power per Euro is still growing by a factor of 100-1000 per decade, greatly increasing the feasibility of neural networks in general, which have started to yield humancompetitive results in challenging pattern recognition competitions. Finally, a recent formal theory of fun and creativity identifies basic principles of curious and creative machines, laying foundations for artificial scientists and artists. Here I will briefly review some of the new results of my lab at IDSIA, and speculate about future developments, pointing out that the time intervals between the most notable events in over 40,000 years or 2 9 lifetimes of human history have sped up exponentially, apparently converging to zero within the next few decades. Or is this impression just a by-product of the way humans allocate memory space to past events? Note: this is the 2012 update of a 2007 publication [73]. Compare also the 2006 celebration of 75 years of AI [71].

show abstract

Section: Recurrent / Deep Neural Networkmentioning

confidence: 99%

“…Our RL RNN now outperform many previous methods on benchmarks [21], creating memories of important events and solving numerous tasks unsolvable by classical RL methods. Several best paper awards resulted from this research, e.g., [18,92].…”

Section: Recurrent / Deep Neural Networkmentioning

confidence: 99%

New Millennium AI and the Convergence of History: Update of 2012

Schmidhuber

2012

The Frontiers Collection

View full text Add to dashboard Cite

show abstract

“…Natural evolution strategies (NES) [3,[8][9][10][11] are a class of evolutionary algorithms for real-valued optimization. They maintain a Gaussian search distribution with fully adaptive covariance matrix.…”

Section: Natural Evolution Strategiesmentioning

confidence: 99%

“…The recently introduced family of natural evolution strategies (NES [3,[8][9][10][11]), consists in an optimization method that follows a sampled natural gradient of the expected fitness, and as such, provides a more principled alternative to CMA-ES. In this paper we combine the well-founded framework of NES with the proven approach of tackling MOO using evolution strategies.…”

Section: Introductionmentioning

confidence: 99%

A Natural Evolution Strategy for Multi-objective Optimization

Glasmachers

Schaul

Schmidhuber

2010

Parallel Problem Solving From Nature, PPSN XI

Self Cite

View full text Add to dashboard Cite

Abstract. The recently introduced family of natural evolution strategies (NES), a novel stochastic descent method employing the natural gradient, is providing a more principled alternative to the well-known covariance matrix adaptation evolution strategy (CMA-ES). Until now, NES could only be used for single-objective optimization. This paper extends the approach to the multi-objective case, by first deriving a (1 + 1) hillclimber version of NES which is then used as the core component of a multi-objective optimization algorithm. We empirically evaluate the approach on a battery of benchmark functions and find it to be competitive with the state-of-the-art.

show abstract

“…Modern, flexible, and easy to implement natural evolution strategies [31,28,27,11,10,25] are used as search algorithms for kernel representations of continuous functions of a single variable (time). The efficiency is demonstrated on the problem of finding kinematic plan representations for redundant robot arms movement.…”

Section: Introductionmentioning

confidence: 99%

Kernel representations for evolving continuous functions

2012

View full text Add to dashboard Cite

Abstract. To parameterize continuous functions for evolutionary learning, we use kernel expansions in nested sequences of function spaces of growing complexity. This approach is particularly powerful when dealing with non-convex constraints and discontinuous objective functions. Kernel methods offer a number of beneficial properties for parameterizing continuous functions, such as smoothness and locality, which make them attractive as a basis for mutation operators. Beyond such practical considerations, kernel methods make heavy use of inner products in function space and offer a well established regularization framework. We show how evolutionary computation can profit from these properties. Searching function spaces of iteratively increasing complexity allows the solution to evolve from a simple first guess to a complex and highly refined function. At transition points where the evolution strategy is confronted with the next level of functional complexity, the kernel framework can be used to project the search distribution into the extended search space. The feasibility of the method is demonstrated on challenging trajectory planning problems where redundant robots have to avoid obstacles.

show abstract

Efficient natural evolution strategies

Cited by 83 publications

References 7 publications

New Millennium AI and the Convergence of History: Update of 2012

New Millennium AI and the Convergence of History: Update of 2012

A Natural Evolution Strategy for Multi-objective Optimization

Kernel representations for evolving continuous functions

Contact Info

Product

Resources

About