Evolution strategies for continuous optimization: A survey of the state-of-the-art

Li, Zhenhua; Lin, Xi Victoria; Zhang, Qingfu; Liu, Hailin

doi:10.1016/j.swevo.2020.100694

“…Several solutions have been proposed to improve noise handling in ESs, such as re-evaluation of points [305] and adapting the population size during fitness evaluation to improve the signal-to-noise ratio [306]. A detailed summary of the challenges related to ESs, such as differential evolution and swarm optimization, is presented in [21].…”

Section: B Evolution Strategiesmentioning

confidence: 99%

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Majid¹,

Saaybi²,

Rietbergen³

et al. 2021

Preprint

View full text Add to dashboard Cite

<div>Deep Reinforcement Learning (DRL) has the potential to surpass human-level control in sequential decision-making problems. Evolution Strategies (ESs) have different characteristics than DRL, yet they are promoted as a scalable alternative. </div><div>To get insights into their strengths and weaknesses, in this paper, we put the two approaches side by side. After presenting the fundamental concepts and algorithms for each of the two approaches, they are compared from the perspectives of scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the paper discusses hybrid algorithms, combining aspects of both DRL and ESs, and how they attempt to capitalize on the benefits of both techniques. Lastly, both approaches are compared based on the set of applications they support, showing their potential for tackling real-world problems.</div><div>This paper aims to present an overview of how DRL and ESs can be used, either independently or in unison, to solve specific learning tasks. It is intended to guide researchers to select which method suits them best and provides a bird's eye view of the overall literature in the field. Further, we also provide application scenarios and open challenges. </div>

show abstract

“…Evolution Strategies (ESs) are set of a population-based black-box optimization algorithms often applied to continuous search spaces problems to find the optimal solutions [20,21]. ESs do not require modeling the problem as an MDP, neither the objective function f (x) has to be differentiable and continuous.…”

Section: B Evolution Strategiesmentioning

confidence: 99%

“…Noise handling. While ESs tolerate some noise due to their randomized nature, noise renders their computations more difficult and causes their performance to approach random walk [21,305,306]. Several solutions have been proposed to improve noise handling in ESs, such as re-evaluation of points [305] and adapting the population size during fitness evaluation to improve the signal-to-noise ratio [306].…”

Section: B Evolution Strategiesmentioning

confidence: 99%

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Majid¹

2021

Preprint

3

0

View full text Add to dashboard Cite

<div>Deep Reinforcement Learning (DRL) has the potential to surpass human-level control in sequential decision-making problems. Evolution Strategies (ESs) have different characteristics than DRL, yet they are promoted as a scalable alternative. </div><div>To get insights into their strengths and weaknesses, in this paper, we put the two approaches side by side. After presenting the fundamental concepts and algorithms for each of the two approaches, they are compared from the perspectives of scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the paper discusses hybrid algorithms, combining aspects of both DRL and ESs, and how they attempt to capitalize on the benefits of both techniques. Lastly, both approaches are compared based on the set of applications they support, showing their potential for tackling real-world problems.</div><div>This paper aims to present an overview of how DRL and ESs can be used, either independently or in unison, to solve specific learning tasks. It is intended to guide researchers to select which method suits them best and provides a bird's eye view of the overall literature in the field. Further, we also provide application scenarios and open challenges. </div>

show abstract

“…It is a set of evolutionary techniques used for the development of robotic systems, as stated by Nolfi [2] in his book, which delivered the theoretical basics for this topic. ER strategies were described in a survey by Back [3] and in a newer paper by Li [4]. ER is often used to synthesize robot controllers for a specific behavior [5] or multiple behaviors [6].…”

Section: Introductionmentioning

confidence: 99%

Genetic Optimization of a Manipulator: Comparison between Straight, Rounded, and Curved Mechanism Links

Pastor

¹

,

Bobovský

²

,

Huczala

³

et al. 2021

View full text Add to dashboard Cite

There are several ubiquitous kinematic structures that are used in industrial robots, with the most prominent being a six-axis angular structure. However, researchers are experimenting with task-based mechanism synthesis that could provide higher efficiency with custom optimized manipulators. Many studies have focused on finding the most efficient optimization algorithm for task-based robot manipulators. These manipulators, however, are usually optimized from simple modular joints and links, without exploring more elaborate modules. Here, we show that link modules defined by small numbers of parameters have better performance than more complicated ones. We compare four different manipulator link types, namely basic predefined links with fixed dimensions, straight links that can be optimized for different lengths, rounded links, and links with a curvature defined by a Hermite spline. Manipulators are then built from these modules using a genetic algorithm and are optimized for three different tasks. The results demonstrate that manipulators built from simple links not only converge faster, which is expected given the fewer optimized parameters, but also converge on lower cost values.

show abstract

Evolution strategies for continuous optimization: A survey of the state-of-the-art

Cited by 64 publications

References 93 publications

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

Genetic Optimization of a Manipulator: Comparison between Straight, Rounded, and Curved Mechanism Links

Contact Info

Product

Resources

About