Mapping hearthstone deck spaces through MAP-elites with sliding boundaries

Abstract

Quality diversity (QD) algorithms such as MAP-Elites have emerged as a powerful alternative to traditional single-objective optimization methods. They were initially applied to evolutionary robotics problems such as locomotion and maze navigation, but have yet to see widespread application. We argue that these algorithms are perfectly suited to the rich domain of video games, which contains many relevant problems with a multitude of successful strategies and often also multiple dimensions along which solutions can vary.This paper introduces a novel modification of the MAP-Elites algorithm called MAP-Elites with Sliding Boundaries (MESB) and applies it to the design and rebalancing of Hearthstone, a popular collectible card game chosen for its number of multidimensional behavior features relevant to particular styles of play. To avoid overpopulating cells with conflated behaviors, MESB slides the boundaries of cells based on the distribution of evolved individuals. Experiments in this paper demonstrate the performance of MESB in Hearthstone. Results suggest MESB finds diverse ways of playing the game well along the selected behavioral dimensions. Further analysis of the evolved strategies reveals common patterns that recur across behavioral dimensions and explores how MESB can help rebalance the game.

Procedural Content Generation through Quality Diversity

“…Regarding online expressivity analysis, we may argue that it covers (in some cases implicitly) all the described cases of PCG-QD. For instance, in [13] the diversity characterization is used to inspect the expressive features of the cards evolved. More importantly, the underlying distribution of the cards' mana plays an active role during the evolutionary search by affecting the boundaries of the space partitioning.…”

“…MAP-Elites with Sliding Boundaries [13] was used to generate decks for the online card game Hearthstone (Blizzard, 2014). The main goal is to generate high-performing decks Fig.…”

“…Another direction for future work in games is applying QD algorithms to debug, diagnose and understand game playing agents. QD algorithms can generate a diverse set of levels that capture different important features of the level itself and an agent's gameplay [13], [19]. By analyzing the outputs, we can understand which behavior has the most effect on the performance of the playing agent.…”

The Many AI Challenges of Hearthstone

“…While playing the game given an existing deck has so far received the most attention, there are interesting challenges inherent in the domain of building these decks [4,19,20,21]. Human players often build them through experimentation and the evolving meta strategies of expert players, but automatically creating such decks could potentially lead to a richer diversity of meta strategies.…”

“…Like many optimization algorithms including evolutionary strategies, search is biased toward one or several performance metrics. Examples of such performance metrics are winrate [21] or the difference in health between players [4,11,19]. In single objective optimization, the idea is that the search space will converge to a relatively small yet powerful space of near-optimal decks (shown in figure 1a).…”