Stabilized Nested Rollout Policy Adaptation

Cazenave, Tristan; Sevestre, Jean-Baptiste; Toulemont, Matthieu

doi:10.1007/978-3-030-89453-5_2

Cited by 9 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Beam-NRPA (Cazenave and Teytaud 2012) optima. High-Diversity NPRA (HD-NRPA) (Edelkamp and Cazenave 2016) elaborates on this observation to increase the diversity of the beam, so that according to some specification of distance solutions too close to existing ones are removed from the beam.…”

Section: Monte Carlo Search Frameworkmentioning

confidence: 99%

Solving Graph Optimization Problems in a Framework for Monte-Carlo Search

Edelkamp

Externest

Kuhl

et al. 2021

SOCS

View full text Add to dashboard Cite

show abstract

Section: Monte Carlo Search Frameworkmentioning

confidence: 99%

Solving Graph Optimization Problems in a Framework for Monte-Carlo Search

Edelkamp

Externest

Kuhl

et al. 2021

SOCS

View full text Add to dashboard Cite

show abstract

“…After that, a new Nested Rollout Policy Adaptation algorithm achieved a new 82 steps record [25]. Thereafter, Cazenave applied Beam Nested Rollout Policy Adaptation [26], which reached the same 82 steps record but did not exceed it, indicating the difficulty of making further progress on Morpion Solitaire using traditional search heuristics.…”

Section: Related Workmentioning

confidence: 99%

“…A score of 80 moves was found by means of Nested Monte-Carlo search [24]. In addition, [25] found a new record with 82 steps, and [26] also found a 82 steps solution. It has been proven mathematically that the 5D version has an upper bound of 121 [27].…”

Section: Morpion Solitairementioning

confidence: 99%

Tackling Morpion Solitaire with AlphaZero-like Ranked Reward Reinforcement Learning

Wang

Preuß

Emmerich

et al. 2020

2020 22nd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC)

View full text Add to dashboard Cite

Morpion Solitaire is a popular single player game, performed with paper and pencil. Due to its large state space (on the order of the game of Go) traditional search algorithms, such as MCTS, have not been able to find good solutions. A later algorithm, Nested Rollout Policy Adaptation, was able to find a new record of 82 steps, albeit with large computational resources. After achieving this record, to the best of our knowledge, there has been no further progress reported, for about a decade.In this paper we take the recent impressive performance of deep self-learning reinforcement learning approaches from AlphaGo/AlphaZero as inspiration to design a searcher for Morpion Solitaire. A challenge of Morpion Solitaire is that the state space is sparse, there are few win/loss signals. Instead, we use an approach known as ranked reward to create a reinforcement learning self-play framework for Morpion Solitaire. This enables us to find medium-quality solutions with reasonable computational effort. Our record is a 67 steps solution, which is very close to the human best (68) without any other adaptation to the problem than using ranked reward. We list many further avenues for potential improvement.

show abstract

“…Stabilized NRPA [12] is a simple improvement of NRPA. The principle is to play P playouts at level 1 before each call to the adapt function.…”

Section: Stabilized Gnrpamentioning

confidence: 99%

“…Beam NRPA has already been applied successfully to the TSPTW and to Morpion Solitaire [14]. The best results were obtained using a beam at level 1.…”

Section: Beam Gnrpamentioning

confidence: 99%

Monte Carlo Inverse Folding

Cazenave,

Fournier

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The RNA Inverse Folding problem comes from computational biology. The goal is to find a molecule that has a given folding. It is important for scientific fields such as bioengineering, pharmaceutical research, biochemistry, synthetic biology and RNA nanostructures. Nested Monte Carlo Search has given excellent results for this problem. We propose to adapt and evaluate different Monte Carlo Search algorithms for the RNA Inverse Folding problem.

show abstract

Stabilized Nested Rollout Policy Adaptation

Cited by 9 publications

References 22 publications

Solving Graph Optimization Problems in a Framework for Monte-Carlo Search

Solving Graph Optimization Problems in a Framework for Monte-Carlo Search

Tackling Morpion Solitaire with AlphaZero-like Ranked Reward Reinforcement Learning

Monte Carlo Inverse Folding

Contact Info

Product

Resources

About