Online Evolution for Multi-action Adversarial Games

Abstract

Abstract. We present Online Evolution, a novel method for playing turn-based multi-action adversarial games. Such games, which include most strategy games, have extremely high branching factors due to each turn having multiple actions. In Online Evolution, an evolutionary algorithm is used to evolve the combination of atomic actions that make up a single move, with a state evaluation function used for fitness. We implement Online Evolution for the turn-based multi-action game Hero Academy and compare it with a standard Monte Carlo Tree Search implementation as well as two types of greedy algorithms. Online Evolution is shown to outperform these methods by a large margin. This shows that evolutionary planning on the level of a single move can be very effective for this sort of problems.

Analysis of Vanilla Rolling Horizon Evolution Parameters in General Video Game Playing

“…Their research suggests EAs to be viable algorithms in general environments, and that a deeper exploration should be performed with an emphasis on heuristic improvement. N. Justesen et al [10] used online evolution for action decision in Hero Academy, a game in which each player counts on multiple units to move in a single turn, presenting a branching factor of a million actions. In this study, groups of actions are evolved for a single turn, to be performed by up to 6 different units.…”

MCTS/EA hybrid GVGAI players and game difficulty estimation

“…More recently, the work by Justesen et al [7] shows that RHEA can achieve high performance in Hero Academy, a game with a very large branching factor.…”

“…In this work, we therefore explore different hybridizations, namely (1) integrating parts of the MCTS method into a rolling horizon EA [7,12], and (2) splitting the computation budget between both methods. Both combinations are experimentally shown to perform well, with the first hybrid possessing a small advantage.…”

Deep Learning for Video Game Playing

“…Whereas the average branching factor hovers around 30 for Chess and 300 for Go, a game like StarCraft has a branching factor that is orders of magnitudes larger. While recent advances in evolutionary planning have allowed realtime and long-term planning in games with larger branching factors to [66], [159], [67], how we can scale Deep RL to such levels of complexity is an important open challenge. Learning heuristics with deep learning in these games to enhance search algorithms is also a promising direction.…”