A Survey of Monte Carlo Tree Search Methods

Abstract

Monte Carlo Tree Search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarise the results from the key game and non-game domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

A Two-Stage Online Approach for Collaborative Multi-agent Planning Under Uncertainty

“…UCT (Upper Confidence bounds applied to Trees) [8] is a state-of-the-art anytime algorithm that combines MCTS [3] with multi-bandit selection methods [8], and has been utilised for planning in domains pervaded by uncertainty. UCT [5] allows to quickly return a non-trivial decision after performing a series of rollouts in which outcomes of actions are sampled based on their probability.…”

“…New decision nodes for the outcomes of the newly generated chance node are also added to the tree. The subsequent rollout phase of UCT is modified so that, at each iteration of the algorithm, a number r of rollouts are carried out for each non-terminal outcome 3 . This allows to quickly obtain accurate reward estimates for the state from which rollouts are being currently performed, as well as thoroughly exploring the different courses of action available from each outcome.…”

“…Online planning differentiates itself from offline planning by not fully elaborating a plan before execution, but instead to interleave planning and execution. To this end, it employs approximate methods such as Monte-Carlo Tree Search (MCTS) [3] to return the next 'good enough' action rather than a complete series of actions [5]. online planning approaches are their ability to narrow the scope of the search space, to return "good enough" actions anytime and to efficiently re-plan when an unexpected situation is encountered while acting.…”

Machine Learning-Based Experimental Design in Materials Science

“…Children of the node are scored with different methods. The most common one is the Upper Confidence Bound (UCB) score [20],…”

“…The significant success of Monte Carlo tree search (MCTS) [20] in computer Go game [14] inspired researchers to develop similar approaches in different research areas including other type of games [21][22][23][24]. MCTS is a guided-random best-first search method that models the search space as a gradually expanded tree.…”