Self-adaptive MCTS for General Video Game Playing

Abstract

Monte-Carlo Tree Search (MCTS) has shown particular success in General Game Playing (GGP) and General Video Game Playing (GVGP) and many enhancements and variants have been developed. Recently, an on-line adaptive parameter tuning mechanism for MCTS agents has been proposed that almost achieves the same performance as off-line tuning in GGP. In this paper we apply the same approach to GVGP and use the popular General Video Game AI (GVGAI) framework, in which the time allowed to make a decision is only 40ms. We design three Self-Adaptive MCTS (SA-MCTS) agents that optimize on-line the parameters of a standard non-Self-Adaptive MCTS agent of GVGAI. The three agents select the parameter values using Naïve Monte-Carlo, an Evolutionary Algorithm and an N-Tuple Bandit Evolutionary Algorithm respectively, and are tested on 20 single-player games of GVGAI. The SA-MCTS agents achieve more robust results on the tested games. With the same time setting, they perform similarly to the baseline standard MCTS agent in the games for which the baseline agent performs well, and significantly improve the win rate in the games for which the baseline agent performs poorly. As validation, we also test the performance of non-Self-Adaptive MCTS instances that use the most sampled parameter settings during the on-line tuning of each of the three SA-MCTS agents for each game. Results show that these parameter settings improve the win rate on the games Wait for Breakfast and Escape by 4 times and 150 times, respectively.

Self-Adaptive Rolling Horizon Evolutionary Algorithms for General Video Game Playing

“…Similarly, an on-line mechanism was used to find the best parameter configuration for an MCTS agent depending on the game being played [6], [7]. In this approach, the result of each MCTS simulation is used to evaluate the quality of the parameter values that control the simulation.…”

“…Moreover, statistics collected so far on the performance of parameter values are used to choose which values to evaluate next. This approach has been tested both on classic board games [6] and on arcade-style video games [7]. On board games, it had positive results, especially when the number of tuned parameters is small.…”

“…which combinations of values are legal. There are various allocation strategies that decide which combinations of parameter values should be evaluated, how many times and in which order [6], [7]. The ones considered in this paper are described below.…”

“…For simulation-based agents, like MCTS, previous research proposed a possible way to address this problem, which consists in a method that tunes search-control parameters on-line, i.e. while playing a single run of a game [6], [7]. For evolutionary-based GVGP agents like the ones based on RHEA, however, no attempt at on-line adaptation has been made so far.…”

“…To do so, the on-line tuning method previously proposed for MCTS is adapted. Different allocation strategies have also been proposed in previous work for on-line parameter tuning of MCTS agents [6], [7]. These strategies decide how to distribute the available samples to evaluate the various parameter value combinations.…”