Learning Probabilistic Behavior Models in Real-Time Strategy Games

Abstract

We study the problem of learning probabilistic models of high-level strategic behavior in the real-time strategy (RTS) game StarCraft. The models are automatically learned from sets of game logs and aim to capture the common strategic states and decision points that arise in those games. Unlike most work on behavior/strategy learning and prediction in RTS games, our data-centric approach is not biased by or limited to any set of preconceived strategic concepts. Further, since our behavior model is based on the well-developed and generic paradigm of hidden Markov models, it supports a variety of uses for the design of AI players and human assistants. For example, the learned models can be used to make probabilistic predictions of a player's future actions based on observations, to simulate possible future trajectories of a player, or to identify uncharacteristic or novel strategies in a game database. In addition, the learned qualitative structure of the model can be analyzed by humans in order to categorize common strategic elements. We demonstrate our approach by learning models from 331 expert-level games and provide both a qualitative and quantitative assessment of the learned model's utility.

Predicting Army Combat Outcomes in StarCraft

“…Naturally, PGMs have seen an increased popularity in the RTS domain over the past few years. Hidden Markov Models (a simple type of PGM) have been used to learn high-level strategies from data (Dereszynski et al 2011) and PGMs have been used to predict the opponent's opening strategy (Synnaeve and Bessiere 2011b) and to guess the order that the opponent is building units in Synnaeve and Bessiere (2011a). The same research group has also developed models that allow their RTS agent to make decisions about where on the map it should send units to attack and with what kinds of units (Synnaeve and Bessiere 2012a).…”

Global State Evaluation in StarCraft

“…Which policies are used can be determined by making the bot aware of certain aspects of the opponent. For example, if you have determined that the opponent is implementing strategy A, and you have previously determined that strategy B is a good counter to A, then you can start running strategy B (Dereszynski et al 2011). Likewise, UAlbertaBot (code.google.com/p/ualbertabot), which won last year's AI-IDE StarCraft AI competition, currently uses simulation results to determine if it should engage the opponent in combat scenarios or not.…”

“…Although there has been a recent trend of using StarCraft replays as data for machine learning tasks (Weber and Mateas 2009) (Synnaeve and Bessière 2011) (Dereszynski et al 2011), little work has been regarding state evaluation in RTS games. (Yang, Harrison, and Roberts 2014) tries to predict game outcomes in Massively Online Battle Arena (MOBA) games, a different but similar genre of game to RTS.…”

StarCraft Winner Prediction

“…They found it difficult to determine opponent strategy in the early game. Dereszynski et al (Dereszynski et al 2011) successfully used a statistical model for predicting opponent behavior and strategy in StarCraft.…”