Systematic Selection of N-Tuple Networks for 2048

Oka, Kazuto; Matsuzaki, Kiminori

doi:10.1007/978-3-319-50935-8_8

Cited by 10 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the one hand, the enormous Table V: Performance of the best n-tuple network with respect to the search limit (constant depth or time limit per move). [21] 234 136 3 88 000 n-tuple network, TD(0), 1-ply Wu et al [38] 328 946 11 300 multi-stage TD(0), 3-ply Xiao et al [39] 442 419 32 3 hand-made features, CMA-ES, adaptive depth Yeh et al [40] 443 number of parameters of n-tuple network can be seen as a disadvantage as it requires a vast amount of computer memory. On the other hand, this is also the reason that the lock-free optimistic parallelism can work seamlessly.…”

Section: Discussionmentioning

confidence: 99%

“…Further refinement its controller lead to scoring 443 526 on average [40]. Recently, Oka and Matsuzaki systematically evaluated a different n-tuple shapes to construct a 1-ply controller achieving 234 136 points on average [21]. Rodgers and Levine investigated the use of various tree search techniques for 2048, but were only able to score 140 000 on average [22].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Mastering 2048 With Delayed Temporal Coherence Learning, Multistage Weight Promotion, Redundant Encoding, and Carousel Shaping

Jaśkowski

2018

IEEE Trans. Games

View full text Add to dashboard Cite

Abstract-2048 is an engaging single-player nondeterministic video puzzle game, which, thanks to the simple rules and hard-to-master gameplay, has gained massive popularity in recent years. As 2048 can be conveniently embedded into the discrete-state Markov decision processes framework, we treat it as a testbed for evaluating existing and new methods in reinforcement learning. With the aim to develop a strong 2048 playing program, we employ temporal difference learning with systematic n-tuple networks. We show that this basic method can be significantly improved with temporal coherence learning, multi-stage function approximator with weight promotion, carousel shaping, and redundant encoding. In addition, we demonstrate how to take advantage of the characteristics of the n-tuple network, to improve the algorithmic effectiveness of the learning process by delaying the (decayed) update and applying lock-free optimistic parallelism to effortlessly make advantage of multiple CPU cores. This way, we were able to develop the best known 2048 playing program to date, which confirms the effectiveness of the introduced methods for discrete-state Markov decision problems.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Mastering 2048 With Delayed Temporal Coherence Learning, Multistage Weight Promotion, Redundant Encoding, and Carousel Shaping

Jaśkowski

2018

IEEE Trans. Games

View full text Add to dashboard Cite

show abstract

“…Szubert and Jaśkowski [25] reported that the computer player performed significantly better by introducing 6-tuples instead of 4-tuples. Some studies used larger 7-tuples [11], [14], [20]. [20] and Matsuzaki [14] analyzed the perfor-mance of players that utilizes many 6-tuples or 7-tuples.…”

Section: Players Based On N-tuple Networkmentioning

confidence: 99%

“…Some studies used larger 7-tuples [11], [14], [20]. [20] and Matsuzaki [14] analyzed the perfor-mance of players that utilizes many 6-tuples or 7-tuples. Jaśkowski's redundant encoding [11] is also a technique to increase the number of N-tuples (and we can save the number of weights by using smaller additional tuples).…”

Section: Players Based On N-tuple Networkmentioning

confidence: 99%

Playing Game 2048 with Deep Convolutional Neural Networks Trained by Supervised Learning

Kondo

Matsuzaki

2019

Journal of Information Processing

Self Cite

View full text Add to dashboard Cite

Game 2048 is a stochastic single-player game and development of strong computer players for Game 2048 has been based on N-tuple networks trained by reinforcement learning. Some computer players were developed with (convolutional) neural networks, but their performance was poor. In this study, we develop computer players for Game 2048 based on deep convolutional neural networks (DCNNs). We increment the number of convolution layers from two to nine, while keeping the number of weights almost the same. We train the DCNNs by applying supervised learning with a large number of play records from existing strong computer players. The best average score achieved is 93,830 with five convolution layers, and the best maximum score achieved is 401,912 with seven convolution layers. These results are better than existing neural-network-based players, while our DCNNs have less weights.

show abstract

“…We have generalized this problem also which is unique from other papers. (4) Note the papers, [11][12][13][14][15][16][17][18][19][20][21][22][23] on the game 2048 which are related to the statistical and mathematical analysis of the game 2048 where our aim is to find the winning strategy for 2048 and to generalize it.…”

Section: Introductionmentioning

confidence: 99%