Evolution of swarm robotics systems with novelty search

Abstract

Novelty search is a recent artificial evolution technique that challenges traditional evolutionary approaches. In novelty search, solutions are rewarded based on their novelty, rather than their quality with respect to a predefined objective. The lack of a predefined objective precludes premature convergence caused by a deceptive fitness function. In this paper, we apply novelty search combined with NEAT to the evolution of neural controllers for homogeneous swarms of robots. Our empirical study is conducted in simulation, and we use a common swarm robotics task -aggregation, and a more challenging task -sharing of an energy recharging station. Our results show that novelty search is unaffected by deception, is notably effective in bootstrapping the evolution, can find solutions with lower complexity than fitness-based evolution, and can find a broad diversity of solutions for the same task. Even in non-deceptive setups, novelty search achieves solution qualities similar to those obtained in traditional fitness-based evolution. Our study also encompasses variants of novelty search that work in concert with fitness-based evolution to combine the exploratory character of novelty search with the exploitatory character of objective-based evolution. We show that these variants can further improve the performance of novelty search. Overall, our study shows that novelty search is a promising alternative for the evolution of controllers for robotic swarms.In this section, we first discuss swarm robotics and evolutionary robotics, and the main challenges associated with these fields. We then present novelty search and how it can overcome some of these challenges. We go on to review recently proposed variants of novelty search. We conclude the section with a description of NEAT, the neuroevolution method used in our experiments.

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

“…Including knowledge can be completely misleading [106], and must therefore be handled with care. When properly done, the gain may be significant [19,17,18,4,5,146,147], but more and more task agnostic methods have been proposed that also have a significant impact on ER efficiency [79,72,103,131,177,106]. These new techniques are very encouraging and allows us to think about a future with a truly automated behavior design method.…”

Embodied artificial life at an impasse can evolutionary robotics methods be scaled?

“…Novelty search [27,28] uses measures of behavioral diversity to drive evolution, while other diversity maintenance methods are implemented with goal-directed fitness terms in weighted sums or via the use of multiobjective optimization methods [2].…”

“…The main concern with these methods is that measuring diversity is an open problem, and it is possible that for complex problems, a fairly deep understanding of the solution space is needed [27,78]. Novelty search methods have been used without explicit task oriented fitness functions [8].…”

“…Problems with task-specific fitness functions have become a prominent focus in ER research [2,27,28,70]. This subsection looks at the major classes of fitness functions that were outlined in Section IV, concluding that most forms of task-specific fitness functions are ultimately limited to extended optimization in terms of utility.…”

“…Doncieux and Mouret [1] review recent advances in ER methods in a survey of selective pressures used in evolution. These include diversity maintenance methods [2,[26][27][28] and the use of multiobjective optimization methods [29,30]. Very large populations, situated asynchronous genetic algorithms (GAs) [31] and/or task-independent diversitymaintaining mechanisms coupled with increases in computing power could lead to an order of magnitude increase in the complexity of achievable tasks.…”