Subgoal-Based Temporal Abstraction in Monte-Carlo Tree Search

Gabor, Thomas; Peter, J. Hencil; Phan, Thomy; Meyer, Christian; Linnhoff‐Popien, Claudia

doi:10.24963/ijcai.2019/772

Cited by 11 publications

(10 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different action abstraction Fig. 3 Representation of options within a tree methods can be found in Gabor et al (2019), Moraes et al (2018), and Uriarte and Ontanón (2014).…”

Section: Action Reductionmentioning

confidence: 99%

Monte Carlo Tree Search: a review of recent modifications and applications

Świechowski¹,

et al. 2022

View full text Add to dashboard Cite

Monte Carlo Tree Search (MCTS) is a powerful approach to designing game-playing bots or solving sequential decision problems. The method relies on intelligent tree search that balances exploration and exploitation. MCTS performs random sampling in the form of simulations and stores statistics of actions to make more educated choices in each subsequent iteration. The method has become a state-of-the-art technique for combinatorial games. However, in more complex games (e.g. those with a high branching factor or real-time ones) as well as in various practical domains (e.g. transportation, scheduling or security) an efficient MCTS application often requires its problem-dependent modification or integration with other techniques. Such domain-specific modifications and hybrid approaches are the main focus of this survey. The last major MCTS survey was published in 2012. Contributions that appeared since its release are of particular interest for this review.

show abstract

“…Different action abstraction Fig. 3 Representation of options within a tree methods can be found in Gabor et al (2019), Moraes et al (2018), and Uriarte and Ontanón (2014).…”

Section: Action Reductionmentioning

confidence: 99%

Monte Carlo Tree Search: a review of recent modifications and applications

Świechowski¹,

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Endowing MCTS with both techniques provides more accurate value estimation. Different action abstraction methods can be found in (Gabor et al, 2019;Moraes et al, 2018;Uriarte and Ontanón, 2014).…”

Section: Action Reductionmentioning

confidence: 99%

Monte Carlo Tree Search: A Review of Recent Modifications and Applications

Świechowski,

Godlewski,

Sawicki

et al. 2021

Preprint

View full text Add to dashboard Cite

Monte Carlo Tree Search (MCTS) is a powerful approach to designing game-playing bots or solving sequential decision problems. The method relies on intelligent tree search that balances exploration and exploitation. MCTS performs random sampling in the form of simulations and stores statistics of actions to make more educated choices in each subsequent iteration. The method has become a state-ofthe-art technique for combinatorial games, however, in more complex games (e.g. those with high branching factor or real-time ones), as well as in various practical domains (e.g. transportation, scheduling or security) an efficient MCTS application often requires its problemdependent modification or integration with other techniques. Such domain-specific modifications and hybrid approaches are the main focus of this survey. The last major MCTS survey has been published in 2012. Contributions that appeared since its release are of particular interest for this review.

show abstract

“…Existing approaches based on more powerful subgoal search methods, on the other hand, have their limitations. [9] is perhaps the closest to our method and uses an MCTS to search the subgoal-induced graph. However, it uses a predefined (not learned) predicate function as a subgoal generator, limiting applicability to the problems with available high-quality heuristics.…”

Section: Related Workmentioning

confidence: 99%

Subgoal Search For Complex Reasoning Tasks

Czechowski¹,

Odrzygóźdź²,

Zbysiński³

et al. 2021

Preprint

View full text Add to dashboard Cite

Humans excel in solving complex reasoning tasks through a mental process of moving from one idea to a related one. Inspired by this, we propose Subgoal Search (kSubS) method. Its key component is a learned subgoal generator that produces a diversity of subgoals that are both achievable and closer to the solution. Using subgoals reduces the search space and induces a high-level search graph suitable for efficient planning. In this paper, we implement kSubS using a transformerbased subgoal module coupled with the classical best-first search framework. We show that a simple approach of generating k-th step ahead subgoals is surprisingly efficient on three challenging domains: two popular puzzle games, Sokoban and the Rubik's Cube, and an inequality proving benchmark INT. kSubS achieves strong results including state-of-the-art on INT within a modest computational budget.

show abstract

Subgoal-Based Temporal Abstraction in Monte-Carlo Tree Search

Cited by 11 publications

References 7 publications

Monte Carlo Tree Search: a review of recent modifications and applications

Monte Carlo Tree Search: a review of recent modifications and applications

Monte Carlo Tree Search: A Review of Recent Modifications and Applications

Subgoal Search For Complex Reasoning Tasks

Contact Info

Product

Resources

About