Efficiency of Static Knowledge Bias in Monte-Carlo Tree Search

Ikeda, Kokolo; Viennot, Simon

doi:10.1007/978-3-319-09165-5_3

Cited by 15 publications

(12 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The core MCTS program used for the experiments makes use of technologies like RAVE [9], progressive widening [15], progressive bias [6] and a large amount of knowledge (shape and common fate graph patterns [10]) in the tree search part. On the internet server KGS, where computer programs can play against humans, with the inclusion of adaptive playouts it has reached a rank of 3 dan under the name "abakus".…”

Section: Methodsmentioning

confidence: 99%

Adaptive playouts for online learning of policies during Monte Carlo Tree Search

Graf

Platzner

2016

Theoretical Computer Science

View full text Add to dashboard Cite

Monte-Carlo Tree Search evaluates positions with the help of a playout policy. If the playout policy evaluates a position wrong then there are cases where the tree search has difficulties to find the correct move due to the large search space. This paper explores adaptive playout policies which improve the playout policy during a tree search. With the help of policy gradient reinforcement learning techniques we optimize the playout policy to give better evaluations. We tested the algorithm in Computer Go and measured an increase in playing strength of more than 100 ELO. The resulting program was able to deal with difficult test cases which are known to pose a problem for Monte-Carlo Tree Search.

show abstract

Section: Methodsmentioning

confidence: 99%

Adaptive playouts for online learning of policies during Monte Carlo Tree Search

Graf

Platzner

2016

Theoretical Computer Science

View full text Add to dashboard Cite

show abstract

“…In [8], the experimental results showed increased win rates of 25.1%, 21.5%, and 4.8% respectively in 9 × 9, 13 × 13, and 19 × 19 Go. This technique, with different forms from the above formula, was also successfully applied to Go [9,23,24] and Othello [34]. In [36], the researchers proposed Predictor + UCB (PUCB) which also extended UCB i to introduce biases over moves.…”

Section: Previous Workmentioning

confidence: 95%

“…In practice, count i can be set to many functions, such as n i , √ n i , ln(n i + 1), l i , √ l i , and ln(l i + 1), where l i is the loss count of node i, which is the summation of the number of losses and half the number of draws for CDC in this paper. In [24,34], their formula included the heuristics similar to the one with square root, and in [9] theirs similar to the one with logarithm. However, all the past work [8-10, 23,24,34] considered n i , not l i .…”

Section: Our Workmentioning

confidence: 99%

“…However, with some more knowledge, MCTS can use it to differentiate the newly generated nodes and then guide the search to converge more efficiently. Progressive bias [8][9][10]23,24,34] is one of such techniques, proposed to guide selections based on given knowledge. The researchers incorporated knowledge by adding additional heuristics for the games Go and Othello.…”

Section: Previous Workmentioning

confidence: 99%

See 1 more Smart Citation

An analysis for strength improvement of an MCTS-based program playing Chinese dark chess

Hsueh

Tseng

et al. 2016

Theoretical Computer Science

View full text Add to dashboard Cite

“…First, the exploration in MCTS is guided by the prediction model to lessen the effect of the large branching factor of the game of Go. To focus the search either progressive widening [3] [11], which only searches a limited amount of moves, or progressive bias [11] [12], which artificially increases the value of good Fig. 3.…”

mentioning

confidence: 99%

Common fate graph patterns in Monte Carlo Tree Search for computer go

Graf

Platzner

2014

2014 IEEE Conference on Computational Intelligence and Games

View full text Add to dashboard Cite

In Monte Carlo Tree Search often extra knowledge in form of patterns is used to guide the search and improve the playouts. Shape patterns, which are frequently used in Computer Go, do not describe tactical situations well, so that this knowledge has to be added manually. This is a tedious process which cannot be avoided as it leads to big improvements in playing strength. The common fate graph, which is a special graphical representation of the board, provides an alternative which handles tactical situations much better. In this paper we use the results of linear time graph kernels to extract features from the common fate graph and use them in a Bradley-Terry model to predict expert moves. We include this prediction model into the tree search and the playout part of a Go program using Monte Carlo Tree Search. This leads to better prediction rates and an improvement in playing strength of about 190 ELO.

show abstract

Efficiency of Static Knowledge Bias in Monte-Carlo Tree Search

Cited by 15 publications

References 8 publications

Adaptive playouts for online learning of policies during Monte Carlo Tree Search

Adaptive playouts for online learning of policies during Monte Carlo Tree Search

An analysis for strength improvement of an MCTS-based program playing Chinese dark chess

Common fate graph patterns in Monte Carlo Tree Search for computer go

Contact Info

Product

Resources

About