CLASSQ-L: A Q-Learning Algorithm for Adversarial Real-Time Strategy Games

2021

Game tree search in games with large branching factors is a notoriously hard problem. In this paper, we address this problem with a new sampling strategy for Monte Carlo Tree Search (MCTS) algorithms, called "Naive Sampling", based on a variant of the Multi-armed Bandit problem called the "Combinatorial Multi-armed Bandit" (CMAB) problem. We present a new MCTS algorithm based on Naive Sampling called NaiveMCTS, and evaluate it in the context of real-time strategy (RTS) games. Our results show that as the branching factor grows, NaiveMCTS performs significantly better than other algorithms.

Section: Related Workmentioning

confidence: 99%

The Combinatorial Multi-Armed Bandit Problem and Its Application to Real-Time Strategy Games

2021

“…To palliate this problem several approaches have been explored such as portfolio approaches (Chung, Buro, and Schaeffer 2005), abstracting the action space (Balla and Fern 2009), hierarchical search (Stanescu, Barriga, and Buro 2014), adversarial HTN planning (Ontañón and Buro 2015) or exploration strategies for combinatorial action spaces (Ontañón 2013). All of the previous approaches, however, share the fact that they assume that the system has access to either a forward model of the domain (in order to apply planning or game tree search), or that the system is allowed to use the actual game to run simulations (e.g., (Jaidee and Muñoz-Avila 2012)). The work presented in this paper differs in that we do not assume that the system has access to a completely defined forward model or simulator, but just to a rough definition of the effect of the actions in the game.…”

Section: Real-time Strategy Gamesmentioning

confidence: 99%

Planning in RTS Games with Incomplete Action Definitions via Answer Set Programming

Balduccini

Uriarte

2021

Standard game tree search algorithms, such as minimax or Monte Carlo Tree Search, assume the existence of an accurate forward model that simulates the effects of actions in the game. Creating such model, however, is a challenge in itself.One cause of the complexity of the task is the gap in level of abstraction between the informal specification of the model and its implementation language. To overcome this issue, we propose a technique for the implementation of forward models that relies on the Answer Set Programming paradigm and on well-established knowledge representation techniques from defeasible reasoning and reasoning about actions and change. We evaluate our approach in the context of Real-Time Strategy games using a collection of StarCraft scenarios.

“…Ontañón (2013) presented a MCTS algorithm called NaïveMCTS specifically designed for RTS games, and showed it could handle full-game, but in the context of a simple RTS game. Some work has been done also using Genetic Algorithms and High Climbing methods (Liu, Louis, and Nicolescu 2013) or Reinforcement Learning (Jaidee and Muñoz-Avila 2012).…”

Section: Introductionmentioning

confidence: 99%

Game-Tree Search over High-Level Game States in RTS Games

Uriarte

2021

From an AI point of view, Real-Time Strategy (RTS) games are hard because they have enormous state spaces, they are real-time and partially observable. In this paper, we present an approach to deploy game-tree search in RTS games by using game state abstraction. We propose a high-level abstract representation of the game state, that significantly reduces the branching factor when used for game-tree search algorithms. Using this high-level representation, we evaluate versions of alpha-beta search and of Monte Carlo Tree Search (MCTS). We present experiments in the context of StarCraft showing promising results in dealing with the large branching factors present in RTS games.