Covariance matrix adaptation for the rapid illumination of behavior space

Fontaine, Matthew C.; Togelius, Julian; Nikolaidis, Stefanos; Hoover, Amy K.

doi:10.1145/3377930.3390232

Cited by 86 publications

(88 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This divergent search methodology may cause inefficiencies or failure to learn. Even in problems where only a few parameters are optimized, the GA variation causes a slow convergence [20]. In problems that require behaviors to be encoded by DNNs with a large number of parameters, MAP-Elites with a GA variation typically fails to find the locally optimal behaviors due to a lack of directed search power [7].…”

Section: Map-elites Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Policy gradient assisted MAP-Elites

Nilsson

Cully

2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

Quality-Diversity optimization algorithms such as MAP-Elites, aim to generate collections of both diverse and high-performing solutions to an optimization problem. MAP-Elites has shown promising results in a variety of applications. In particular in evolutionary robotics tasks targeting the generation of behavioral repertoires that highlight the versatility of robots. However, for most robotics applications MAP-Elites is limited to using simple open-loop or low-dimensional controllers. Here we present Policy Gradient Assisted MAP-Elites (PGA-MAP-Elites), a novel algorithm that enables MAP-Elites to efficiently evolve large neural network controllers by introducing a gradient-based variation operator inspired by Deep Reinforcement Learning. This operator leverages gradient estimates obtained from a critic neural network to rapidly find higher-performing solutions and is paired with a traditional genetic variation to maintain a divergent search behavior. The synergy of these operators makes PGA-MAP-Elites an efficient yet powerful algorithm for finding diverse and high-performing behaviors. We evaluate our method on four different tasks for building behavioral repertoires that use deep neural network controllers. The results show that PGA-MAP-Elites significantly improves the quality of the generated repertoires compared to existing methods. CCS CONCEPTS• Computing methodologies → Evolutionary robotics.

show abstract

Section: Map-elites Algorithmmentioning

confidence: 99%

“…Typically the number of optimized parameters is kept below 100 [13,15,40]. GAs are also inefficient [12,20] and prone to finding unstable solutions located on narrow peaks in the optimization landscape that are not repeatable in stochastic environments [14,19,25].…”

mentioning

confidence: 99%

Policy gradient assisted MAP-Elites

Nilsson

Cully

2021

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

show abstract

“…Most interestingly, this approach avoids the need to encode all the requirements in the fitness function, which is required when using traditional optimization algorithms. So far, promising results have been published for designing three-dimensional shapes of aerodynamic bikes (Gaier et al, 2018) ( Figure 2C), to generate content for different games (Fontaine et al, 2020;Gravina et al, 2019), to design molecules (Verhellen and Van den Abeele, 2020), to solve workforce scheduling and routing problems (Urquhart and Hart, 2018), and to find adversarial examples for deep neural networks (Nguyen et al, 2015) or malware (Babaagba et al, 2020).…”

Section: Ll Open Accessmentioning

confidence: 99%

Evolving the Behavior of Machines: From Micro to Macroevolution

Mouret

2020

iScience

View full text Add to dashboard Cite

Evolution gave rise to creatures that are arguably more sophisticated than the greatest human-designed systems. This feat has inspired computer scientists since the advent of computing and led to optimization tools that can evolve complex neural networks for machines-an approach known as ''neuroevolution.'' After a few successes in designing evolvable representations for high-dimensional artifacts, the field has been recently revitalized by going beyond optimization: to many, the wonder of evolution is less in the perfect optimization of each species than in the creativity of such a simple iterative process, that is, in the diversity of species. This modern view of artificial evolution is moving the field away from microevolution, following a fitness gradient in a niche, to macroevolution, filling many niches with highly different species. It already opened promising applications, like evolving gait repertoires, video game levels for different tastes, and diverse designs for aerodynamic bikes.

show abstract

“…Thus, some MAP-Elites variation skip the grid architecture and focus on reducing the amount of feature dimensions or enabling the use of higher dimensions such as Centroidal Voronoi Tessellation-MAP-Elites [98] or Cluster-Elites [99]. Further, the Covariance Matrix Adaptation MAP-Elites algorithm combines the effective adaptive search of Covariance Matrix Adaptation Evolution Strategy with a map of elites, yielding large improvements for real-valued representations in terms of both objective value and number of elites discovered [100]. The work by Fontaine et al was expanded into the Multi-Emitter MAP-Elites, improving the quality, diversity, and convergence speed of MAP-Elites in general [101].…”

Section: One Major Challenge With Map-elites Is the Curse Of Dimensiomentioning

confidence: 99%

“…However, their distinction is mainly on the individuals' encoding, i.e., how each individual or solution is internally represented, which also limits what variant operators can be applied. For instance, Genetic Algorithms use genes encoded as finite strings, while Evolution Strategies' individuals are represented as real-valued vectors where approaches such as Covariance Matrix Adaptation could be applied [100]. Genetic algorithms are used for the work presented in this thesis.…”

Section: Evolutionary Computationmentioning

confidence: 99%

Exploring the Dynamic Properties of Interaction in Mixed-Initiative Procedural Content Generation

Alvarez¹

View full text Add to dashboard Cite

As AI develops, grows, and expands, the more benefits we can have from it. AI is used in multiple fields to assist humans, such as object recognition, self-driving cars, or design tools. However, AI could be used for more than assisting humans in their tasks. It could be employed to collaborate with humans as colleagues in shared tasks, which is usually described as Mixed-Initiative (MI) paradigm. This paradigm creates an interactive scenario that leverage on AI and human strengths with an alternating and proactive initiative to approach a task. However, this paradigm introduces several challenges. For instance, there must be an understanding between humans and AI, where autonomy and initiative become negotiation tokens. In addition, control and expressiveness need to be taken into account to reach some goals. Moreover, although this paradigm has a broader application, it is especially interesting for creative tasks such as games, which are mainly created in collaboration. Creating games and their content is a hard and complex task, since games are content-intensive, multi-faceted, and interacted by external users. Therefore, this thesis explores MI collaboration between human game designers and AI for the co-creation of games, where the AI's role is that of a colleague with the designer. The main hypothesis is that AI can be incorporated in systems as a collaborator, enhancing design tools, fostering human creativity, reducing their workload, and creating adaptive experiences. Furthermore, This collaboration arises several dynamic properties such as control, expressiveness, and initiative, which are all central to this thesis. Quality-Diversity algorithms combined with control mechanisms and interactions for the designer are proposed to investigate this collaboration and properties. Designer and Player modeling is also explored, and several approaches are proposed to create a better workflow, establish adaptive experiences, and enhance the interaction. Through this, it is demonstrated the potential and benefits of these algorithms and models in the MI paradigm.

show abstract

Covariance matrix adaptation for the rapid illumination of behavior space

Cited by 86 publications

References 49 publications

Policy gradient assisted MAP-Elites

Policy gradient assisted MAP-Elites

Evolving the Behavior of Machines: From Micro to Macroevolution

Exploring the Dynamic Properties of Interaction in Mixed-Initiative Procedural Content Generation

Contact Info

Product

Resources

About