Progressive Strategies for Monte-Carlo Tree Search

Chaslot, Guillaume; WINANDS, M.H.M.; Uiterwijk, Jos W. H. M.; HERIK, H.J. VAN DEN; BOUZY, B.

doi:10.1142/9789812709677_0246

Cited by 117 publications

(142 citation statements)

References 0 publications

Supporting

Mentioning

137

Contrasting

Order By: Relevance

“…First, as well as for humans, all time scales of learning are important: offline knowledge (strategic rules and patterns) as in [7,5]; online information (i.e. analysis of a sequence by mental simulations) [10]; transient information (extrapolation as a guide for exploration).…”

Section: Resultsmentioning

confidence: 99%

“…Monte-Carlo Tree Search (MCTS [5,7,11]) is a recent tool for difficult planning tasks. Impressive results have already been produced in the case of the game of Go [7,10].…”

Section: Introductionmentioning

confidence: 99%

“…For estimating which situation should be further analyzed, several algorithms have been proposed: UCT [11] (Upper Confidence Trees), focuses on the proportion of winning simulation plus an uncertainty measure; AMAF [4,1,10] (All Moves As First, also termed RAVE for Rapid Action-Value Estimates in the MCTS context), focuses on a compromise between UCT and heuristic information extracted from the simulations; BAST [6] (Bandit Algorithm for Search in Tree), uses UCB-like bounds modified through the overall number of nodes in the tree. Other related algorithms have been proposed as in [5], essentially using a decreasing impact of a heuristic (pattern-dependent) bias as the number of simulations increases. In all these cases, the idea is to bias random simulations thanks to statistics.…”

Section: Introductionmentioning

confidence: 99%

“…end while Return the move simulated most often from the root of T . The reader is referred to [5,7,11,10,9] for a detailed presentation of MCTS techniques and various scores. We will propose our current bandit formula in section 2.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search

Chaslot

Fiter

Hoock

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We present a new exploration term, more efficient than classical UCT-like exploration terms and combining efficiently expert rules, patterns extracted from datasets, All-Moves-As-First values and classical online values. As this improved bandit formula does not solve several important situations (semeais, nakade) in computer Go, we present three other important improvements which are central in the recent progress of our program MoGo:-We show an expert-based improvement of Monte-Carlo simulations for nakade situations; we also emphasize some limitations of this modification. -We show a technique which preserves diversity in the Monte-Carlo simulation, which greatly improves the results in 19x19. -Whereas the UCB-based exploration term is not efficient in MoGo, we show a new exploration term which is highly efficient in MoGo. MoGo recently won a game with handicap 7 against a 9Dan Pro player, Zhou JunXun, winner of the LG Cup 2007, and a game with handicap 6 against a 1Dan pro player, Li-Chen Chien. 1

show abstract

Section: Resultsmentioning

confidence: 99%

“…Monte-Carlo Tree Search (MCTS [5,7,11]) is a recent tool for difficult planning tasks. Impressive results have already been produced in the case of the game of Go [7,10].…”

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search

Chaslot

Fiter

Hoock

et al. 2010

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…It has been greatly improved by including Progressive Widening and Double Progressive Widening [6,2], RAVE values [7], Blind Values [4], and handcrafted Monte-Carlo moves [17,10]. A crucial component is the Monte-Carlo move generator, also known as the playout generator.…”

Section: Introductionmentioning

confidence: 99%

Learning a Move-Generator for Upper Confidence Trees

Couëtoux

Teytaud

Doghmen

2013

Advances in Intelligent Systems and Applications - Volume 1

View full text Add to dashboard Cite

Abstract. We experiment the introduction of machine learning tools to improve Monte-Carlo Tree Search. More precisely, we propose the use of Direct Policy Search, a classical reinforcement learning paradigm, to learn the Monte-Carlo Move Generator. We experiment our algorithm on different forms of unit commitment problems, including experiments on a problem with both macrolevel and microlevel decisions.

show abstract

AI for General Strategy Game Playing

Nielsen

Jensen

Mahlmann

et al. 2014

Handbook of Digital Games

View full text Add to dashboard Cite

List of Figures vii List of Tables ix Acknowledgments xi AI for General Strategy Game Playing 13 10.1 Introduction 13 10.1.1 Strategy games 13 10.1.2 AI for board games 15 10.1.3 AI for strategy games 16 10.2 Research questions and methodology 18 10.2.1 The SGDL and its framework 19 10.3 The game, the agents and the assigner 21 10.3.1 Implementation details 24 10.4 Agents 24 10.4.1 Random action selection 25 10.4.2 Finite-state machine 26 10.4.3 Neuroevolution of augmenting topologies 26 v vi CONTENTS 10.4.4 MinMax 10.4.5 Monte Carlo Tree Search 10.4.6 Potential fields 10.4.7 Classifier systems 10.5 Results of agent versus agent testing 10.6 Results of human play testing 10.7 Discussion 10.8 Conclusions References LIST OF FIGURES 10.1 A typical screenshot from our example game "RockWars" 10.2 The Commander framework used for the agents in the study. 10.3 Client-server system used for training and experimentation. 10.4 Finite-state automata of the SemiRandom agent units 10.5 Finite-state automaton of the FSM agent's units 10.6 Action tree used to find MultiActions 10.7 Action tree illustrating the use of buildings. 10.8 Summary of agent versus agent results 10.9 Summary of human play results vii LIST OF TABLES 10.1 Unit properties for SGDL models 10.2 Agents in the study 10.3 Summary of agent versus agent results 10.4 Standard deviantions of agent versus agent results 10.5 Summary of human play results ix ACKNOWLEDGMENTS For more technical details, please refer to the first two authors' joint masters thesis [1], or Tobias Mahlmann's PhD thesis including the detailed description of the Strategy Games Description Language [2]. Both of these are available online at http://game.itu.dk/sgdl. This research was supported in part by the Danish Research Agency project "AGameComIn" (274-09-0083). xi CHAPTER 10 AI FOR GENERAL STRATEGY GAME PLAYING 1 For brevity, we will refer to computer strategy games as strategy games in the following sections. AI for General Strategy Game Playing.

show abstract

Progressive Strategies for Monte-Carlo Tree Search

Cited by 117 publications

References 0 publications

Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search

Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search

Learning a Move-Generator for Upper Confidence Trees

AI for General Strategy Game Playing

Contact Info

Product

Resources

About