Information Set Monte Carlo Tree Search

Cowling, Peter I.; Powley, Edward J.; Whitehouse, Daniel

doi:10.1109/tciaig.2012.2200894

Cited by 127 publications

(125 citation statements)

References 20 publications

Supporting

Mentioning

123

Contrasting

Order By: Relevance

“…Another relevant part of AI literature deals with Monte Carlo Tree Search (MTCS) applied to imperfect information games, in which PIMC's top-level Monte Carlo search is replaced by UCT [10] variants acting on information sets, rather than game states [11], [12], [13], [14]. We will discuss how UCT is generalized to searching over information sets in the experimental section, where we compare it with our new recursive imperfect information Monte Carlo search method.…”

Section: R Wmentioning

confidence: 99%

“…The concept of running MCTS on information setswhich collect game states the player to move can't distinguish -is quite similar to IIMC [11], [13]. In this setting the usual nodes of the MCTS tree instead correspond to information sets, and playouts involve sampling a world at the root and then playing the MCTS move for previously observed states.…”

Section: I I M C Smentioning

confidence: 99%

“…The first to study this question were Frank and Basin [4], who identified two types of problems PIMC search suffers from: strategy fusion (i.e., cherry-picking moves in nodes belonging to the same information set), and non-local dependencies between potentially distant nodes in the tree. Long et al[9] present arguments why PIMC search is very successful in games in which, during the course of the game, more and more information is revealed (like in trick-based card games), but doesn't perform as well in other imperfect information games.Another relevant part of AI literature deals with Monte Carlo Tree Search (MTCS) applied to imperfect information games, in which PIMC's top-level Monte Carlo search is replaced by UCT [10] variants acting on information sets, rather than game states [11], [12], [13], [14]. We will discuss how UCT is generalized to searching over information sets in the experimental section, where we compare it with our new recursive imperfect information Monte Carlo search method.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Recursive Monte Carlo search for imperfect information games

Furtak

Buro

2013

2013 IEEE Conference on Computational Inteligence in Games (CIG)

View full text Add to dashboard Cite

Abstract-Perfect information Monte Carlo (PIMC) search is the method of choice for constructing strong AI systems for trick-taking card games. PIMC search evaluates moves in imperfect information games by repeatedly sampling worlds based on state inference and estimating move values by solving the corresponding perfect information scenarios. PIMC search performs well in trick-taking card games despite the fact that it suffers from the strategy fusion problem, whereby the game's information set structure is ignored because moves are evaluated opportunistically in each world. In this paper we describe imperfect information Monte Carlo (IIMC) search, which aims at mitigating this problem by basing move evaluation on more realistic playout sequences rather than perfect information move values. We show that RecPIMC -a recursive IIMC search variant based on perfect information evaluation -performs considerably better than PIMC search in a large class of synthetic imperfect information games and the popular card game of Skat, for which PIMC search is the state-of-the-art cardplay algorithm. I. IIn recent years AI research in the area of imperfect information games has flourished. For instance, in 2008 Poker program "Polaris" defeated a group of six strong human players in two-player limit Texas hold'em in a duplicate match setting [1], and in 2009 "Kermit" reached expert playing strength in Skat, a popular trick-based card game similar to Bridge [2]. The considerable progress in Poker is due to new techniques such as counter factual regret minimization for approximating Nash equilibrium strategies in smaller, abstract versions of the game [3], whereas in Skat fast perfect information Monte Carlo (PIMC) search combined with explicit state inference and heuristic state evaluation elevated programs to the next level.Both approaches have distinct advantages and disadvantages. For instance, solving abstracted game versions off-line leads to fast on-line move computation because, essentially, available moves only have to be sampled from pre-computed probability distributions. However, finding good game abstractions that allow us to approximate move distributions in the original game well isn't trivial. A property of Poker is that the set of legal moves in each game state doesn't depend on the cards players are holding. Therefore, we only need to consider state abstractions (such as hand-strength bucketing) and we don't have to deal with move abstractions.By contrast, in trick-based card games such as Bridge, Spades, and Skat, legal moves are defined by the cards players hold. Therefore, abstracting such games with the intent of using pre-computed move probabilities is non-trivial. PIMC search deals with this problem by sampling game states in accordance with observed moves and private state information, revealing game states to all players, and then evaluating moves by using search algorithms tailored for the perfect information setting. The obvious drawback is that this method blatantly ignores players' ignorance and, conse...

show abstract

Section: R Wmentioning

confidence: 99%

Section: I I M C Smentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

Recursive Monte Carlo search for imperfect information games

Furtak

Buro

2013

2013 IEEE Conference on Computational Inteligence in Games (CIG)

View full text Add to dashboard Cite

show abstract

“…Inspired by the success of MCTS in perfect information games, the algorithm has recently also been adapted for imperfect information games [3][4][5]. Games with imperfect information are fundamentally more complicated than perfect information games, for several reasons.…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we analyze various selection functions in Information Set Monte Carlo Tree Search (IS-MCTS) [3]. We show that the most commonly used selection function -UCT ( [6]) -does not allow the algorithm to converge to good strategies, and even causes the strategies to get worse with more computation time.…”

Section: Introductionmentioning

confidence: 99%

Alternative Selection Functions for Information Set Monte Carlo Tree Search

Lisý

2014

Acta Polytech

View full text Add to dashboard Cite

Abstract. We evaluate the performance of various selection methods for the Monte Carlo Tree Search algorithm in two-player zero-sum extensive-form games with imperfect information. We compare the standard Upper Confident Bounds applied to Trees (UCT) along with the less common Exponential Weights for Exploration and Exploitation (Exp3) and novel Regret matching (RM) selection in two distinct imperfect information games: Imperfect Information Goofspiel and Phantom Tic-Tac-Toe. We show that UCT after initial fast convergence towards a Nash equilibrium computes increasingly worse strategies after some point in time. This is not the case with Exp3 and RM, which also show superior performance in head-to-head matches.

show abstract

Iterative Tree Search in General Game Playing with Incomplete Information

Chitizadeh

Thielscher

2019

Communications in Computer and Information Science

View full text Add to dashboard Cite

Information Set Monte Carlo Tree Search

Cited by 127 publications

References 20 publications

Recursive Monte Carlo search for imperfect information games

Recursive Monte Carlo search for imperfect information games

Alternative Selection Functions for Information Set Monte Carlo Tree Search

Iterative Tree Search in General Game Playing with Incomplete Information

Contact Info

Product

Resources

About