Monte Carlo Tree Search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarise the results from the key game and non-game domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

No abstract

In this paper, we examine the use of Monte Carlo Tree Search (MCTS) for a variant of one of the most popular and profitable games in the world: the card game Magic: The Gathering (M:TG). The game tree for M:TG has a range of distinctive features, which we discuss here, and has incomplete information through the opponent's hidden cards, and randomness through card drawing from a shuffled deck. We investigate a wide range of approaches that use determinization, where all hidden and random information is assumed known to all players, alongside Monte Carlo Tree Search. We consider a number of variations to the rollout strategy using a range of levels of sophistication and expert knowledge, and decaying reward to encourage play urgency. We examine the effect of utilising various pruning strategies in order to increase the information gained from each determinization, alongside methods that increase the relevance of random choices. Additionally we deconstruct the move generation procedure into a binary yes/no decision tree and apply MCTS to this finer grained decision process. We compare our modifications to a basic MCTS approach for Magic: The Gathering using fixed decks, and show that significant improvements in playing strength can be obtained.

Determinization is a technique for making decisions in games with stochasticity and/or imperfect information by sampling instances of the equivalent deterministic game of perfect information. Monte-Carlo Tree Search (MCTS) is an AI technique that has recently proved successful in the domain of deterministic games of perfect information. This paper studies the strengths and weaknesses of determinization coupled with MCTS on a game of imperfect information, the popular Chinese card game Dou Di Zhu. We compare a "cheating" agent (with access to hidden information) to an agent using determinization with random deals. We investigate the fraction of knowledge that a noncheating agent could possibly infer about opponents' hidden cards. Furthermore, we show that an important source of error in determinization arises since this approach searches a tree that does not truly resemble the game tree for a game with stochasticity and imperfect information. Hence we introduce a novel variant of MCTS that operates directly on trees of information sets and show that our algorithm performs well in precisely those situations where determinization using random deals performs poorly.