MoHex 2.0: A Pattern-Based MCTS Hex Player

Huang, Shih-Chieh; Arneson, Broderick; Hayward, Ryan B.; Müller, Martin; Pawlewicz, Jakub

doi:10.1007/978-3-319-09165-5_6

Cited by 29 publications

(27 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that there is no clear indication in the included plots that further improvement is not possible, it is simply a matter of limited time and diminishing returns. We evaluate NeuroHex by testing against the Monte-Carlo tree search player MoHex [8] [3], currently the world's strongest hexbot. See Figure 5.…”

Section: Resultsmentioning

confidence: 99%

NeuroHex: A Deep Q-learning Hex Agent

Young

Vasan

Hayward

2017

Communications in Computer and Information Science

View full text Add to dashboard Cite

DeepMind's recent spectacular success in using deep convolutional neural nets and machine learning to build superhuman level agents -e.g. for Atari games via deep Q-learning and for the game of Go via Reinforcement Learning -raises many questions, including to what extent these methods will succeed in other domains. In this paper we consider DQL for the game of Hex: after supervised initializing, we use selfplay to train NeuroHex, an 11-layer CNN that plays Hex on the 13×13 board. Hex is the classic two-player alternate-turn stone placement game played on a rhombus of hexagonal cells in which the winner is whomever connects their two opposing sides. Despite the large action and state space, our system trains a Q-network capable of strong play with no search. After two weeks of Q-learning, NeuroHex achieves win-rates of 20.4% as first player and 2.1% as second player against a 1-second/move version of MoHex, the current ICGA Olympiad Hex champion. Our data suggests further improvement might be possible with more training time.

show abstract

Section: Resultsmentioning

confidence: 99%

NeuroHex: A Deep Q-learning Hex Agent

Young

Vasan

Hayward

2017

Communications in Computer and Information Science

View full text Add to dashboard Cite

show abstract

“…This is amplified when combined with Monte Carlo Tree Search (MCTS) (Coulom, 2006;Silver et al, 2016Silver et al, , 2017. Deep neural nets have also been used in Hex (Gao et al, 2017a), yielding MoHex-CNN, which is stronger than MoHex 2.0 (Huang et al, 2013;Pawlewicz et al, 2015) on 13×13 Hex. MoHex-CNN uses policy net output for move prior probabilities during the in-tree phase.…”

Section: Related Workmentioning

confidence: 99%

“…The resulting program defeats MoHex 2.0 even on boardsize 13×13, although its margin of victory there is less than on other boardsizes. MoHex 2.0's playout policy was trained mostly on 13×13 games (Huang et al, 2013), so we conjecture that the low q-head prediction accuracy is insufficient to match the strength of MoHex 2.0 playouts. To verify this, we ran extra tournaments with the q-head in MoHex3H's search disabled, forcing MoHex3H to use the same pattern-based playouts as MoHex 2.0.…”

Section: Combined With Searchmentioning

confidence: 99%

A transferable neural network for Hex

Gao

Yan

Hayward

et al. 2019

ICG

Self Cite

View full text Add to dashboard Cite

The game of Hex can be played on multiple boardsizes. Transferring neural net knowledge learned on one boardsize to other boardsizes is of interest, since deep neural nets usually require large size of high quality data to train, whereas expert games can be unavailable or difficult to generate. In this paper we investigate neural transfer learning in Hex. We show that when only boardsize independent neurons are used, the resulting neural net obtained from training on one base boardsize can effectively generalize-without fine-tuning-to multiple target boardsizes, larger or smaller. When transferring to larger boardsizes, fine-tuning provides faster learning and better performance. The strength of the transferable network can be amplified with search: with a single neural net model trained on games from a base boardsize, we obtain players stronger than MoHex 2.0 on multiple target boardsizes.

show abstract

“…For the second case, where different BF-JL applications share the same BF-JL search among different game-playing programs, each program needs to specify the corresponding behaviour for the search. For example, the three programs, NCTU6 for Connect6, CGI (CGI-LAB, 2015a) for Go, and MoHex (Huang, Arneson, Hayward et al, 2014) for Hex, can be paired with JL-UCT to create three separate BF-JL applications. Since JL-UCT uses win rates during the search, different programs need to provide JL-UCT with their own win rates.…”

Section: Bf-jl Applicationsmentioning

confidence: 99%

“…Third, Monte-Carlo Tree Search (MCTS) (Coulom, 2006;Kocsis and Szepesvári, 2006;Gelly, Wang, Munos et al, 2006) is a best-first search algorithm using Monte-Carlo simulations as state evaluations. It has been successfully applied to Go (Gelly, Wang, Munos et al, 2006;Enzenberger, Müller, Arneson et al, 2010;Gelly and Silver, 2011), Hex (Huang, Arneson, Hayward et al, 2014), General Game Playing (Björnsson and Finnsson, 2009), Backgammon (Van Lishout, and Phantom-Go (Borsboom, Saito, Chaslot et al, 2007).…”

Section: Introductionmentioning

confidence: 99%