Deep surrogate assisted MAP-elites for automated hearthstone deckbuilding

Zhang, Yulun; Fontaine, Matthew C.; Hoover, Amy K.; Nikolaidis, Stefanos

doi:10.1145/3512290.3528718

Cited by 17 publications

(22 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This latent space might correspond to the input context to a Transformer-based generative model of the data space or of data-generating programs. Latent-space optimization [165][166][167][168] and quality-diversity search [169][170][171] methods make use of learned low-dimensional representations and surrogate models in this way, but assume stationary objectives. Moreover, it is currently unclear how royalsocietypublishing.org/journal/rsos R. Soc.…”

Section: Q5 How Do We Determine What Data To Acquire Next Via Active ...mentioning

confidence: 99%

General intelligence requires rethinking exploration

2023

View full text Add to dashboard Cite

We are at the cusp of a transition from ‘learning from data’ to ‘learning what data to learn from’ as a central focus of artificial intelligence (AI) research. While the first-order learning problem is not completely solved, large models under unified architectures, such as transformers, have shifted the learning bottleneck from how to effectively train models to how to effectively acquire and use task-relevant data. This problem, which we frame as exploration , is a universal aspect of learning in open-ended domains like the real world. Although the study of exploration in AI is largely limited to the field of reinforcement learning, we argue that exploration is essential to all learning systems, including supervised learning. We propose the problem of generalized exploration to conceptually unify exploration-driven learning between supervised learning and reinforcement learning, allowing us to highlight key similarities across learning settings and open research challenges. Importantly, generalized exploration is a necessary objective for maintaining open-ended learning processes, which in continually learning to discover and solve new problems, provides a promising path to more general intelligence.

show abstract

Section: Q5 How Do We Determine What Data To Acquire Next Via Active ...mentioning

confidence: 99%

General intelligence requires rethinking exploration

2023

View full text Add to dashboard Cite

show abstract

“…Extensions of MAP-Elites include new archive structures [29], [30] and integrations with surrogate models [31], [32], [33], [34], [35]. MAP-Elites has also been applied to environment generation for human-robot interaction [36], [37] and dataset generation for manipulation [38].…”

Section: Background a Map-elitesmentioning

confidence: 99%

Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealing

Tjanaka¹,

Fontaine²,

Aniruddha³

et al. 2022

Preprint

View full text Add to dashboard Cite

Pre-training a diverse set of robot controllers in simulation has enabled robots to adapt online to damage in robot locomotion tasks. However, finding diverse, highperforming controllers requires specialized hardware and extensive tuning of a large number of hyperparameters. On the other hand, the Covariance Matrix Adaptation MAP-Annealing algorithm, an evolution strategies (ES)-based quality diversity algorithm, does not have these limitations and has been shown to achieve state-of-the-art performance in standard benchmark domains. However, CMA-MAE cannot scale to modern neural network controllers due to its quadratic complexity. We leverage efficient approximation methods in ES to propose three new CMA-MAE variants that scale to very high dimensions. Our experiments show that the variants outperform ES-based baselines in benchmark robotic locomotion tasks, while being comparable with state-of-the-art deep reinforcement learningbased quality diversity algorithms. Source code and videos are available at https://scalingcmamae.github.io Compute archive improvement Adapt and with ES Sample solutions Evaluate policies and insert into archive

show abstract

“…The general framework introduced by SAIL for using surrogate models to assist QDO has been adapted to predict descriptors along with objectives [3] in order to find solutions with specific features and as a component in a larger generative design process that is iteratively applied to narrow in on promising design spaces [30], [32].…”

Section: E Surrogate-assisted Phenotypic Nichingmentioning

confidence: 99%

A discretization-free metric for assessing quality diversity algorithms

Kent

Branke

Gaier

et al. 2022

Proceedings of the Genetic and Evolutionary Computation Conference Companion

View full text Add to dashboard Cite

Quality Diversity (QD) algorithms such as MAP-Elites are a class of optimisation techniques that attempt to find many high performing points that all behave differently according to a user-defined behavioural metric.In this paper we propose the Bayesian Optimisation of Elites (BOP-Elites) algorithm. Designed for problems with expensive black-box fitness and behaviour functions, it is able to return a QD solution-set with excellent final performance already after a relatively small number of samples. BOP-Elites models both fitness and behavioural descriptors with Gaussian Process (GP) surrogate models and uses Bayesian Optimisation (BO) strategies for choosing points to evaluate in order to solve the qualitydiversity problem. In addition, BOP-Elites produces high quality surrogate models which can be used after convergence to predict solutions with any behaviour in a continuous range.An empirical comparison shows that BOP-Elites significantly outperforms other state-of-the-art algorithms without the need for problem-specific parameter tuning.

show abstract

Deep surrogate assisted MAP-elites for automated hearthstone deckbuilding

Cited by 17 publications

References 20 publications

General intelligence requires rethinking exploration

General intelligence requires rethinking exploration

Training Diverse High-Dimensional Controllers by Scaling Covariance Matrix Adaptation MAP-Annealing

A discretization-free metric for assessing quality diversity algorithms

Contact Info

Product

Resources

About