Playing Game 2048 with Deep Convolutional Neural Networks Trained by Supervised Learning

Abstract

Game 2048 is a stochastic single-player game and development of strong computer players for Game 2048 has been based on N-tuple networks trained by reinforcement learning. Some computer players were developed with (convolutional) neural networks, but their performance was poor. In this study, we develop computer players for Game 2048 based on deep convolutional neural networks (DCNNs). We increment the number of convolution layers from two to nine, while keeping the number of weights almost the same. We train the DCNNs by applying supervised learning with a large number of play records from existing strong computer players. The best average score achieved is 93,830 with five convolution layers, and the best maximum score achieved is 401,912 with seven convolution layers. These results are better than existing neural-network-based players, while our DCNNs have less weights.

A review on the application of reinforcement learning to the 2048 game

“…They can study complex patterns and relationships from data. In the study by Kondo et al [2], a convolutional neural network (CNN) was trained to play 2048. A CNN is a type of neural network that is particularly effective at processing images and other multidimensional data.…”

MAIDCRL: Semi-centralized Multi-Agent Influence Dense-CNN Reinforcement Learning

“…Nair et al introduced parallel learning with a distributed network and a shared memory replay to split the learning tasks across multiple instances of simulations, effectively increasing the exploration speed at which agents learn [3]. Liu et al modeled the problem of MARL for a constrained, partially observable [8] where they adopted supervised learning with multiple layers of convolutional networks and found exceptional scores. Other notable applications which enabled us to further investigate the broad spectrum of CNN are checking electricity prices, multi-microgrid co-operative systems, mimicking Go Experts [9] etc.…”

“…Stanescu et al presented a CNN for RTS game state evaluation that goes beyond commonly used material-based evaluations by also taking spatial relations between units into account [7]. A Deep convolutional neural network (DCNN) has been developed by Kondo and Matsuzaki[8] where they adopted supervised learning with multiple layers of convolutional networks and found exceptional scores. Other notable applications which enabled us to further investigate the broad spectrum of CNN are checking electricity prices, multi-microgrid co-operative systems, mimicking Go Experts[9] etc.…”

2048 Oyunu İçi̇n Farkli Algori̇tmalarin Performans Anali̇zi̇

“…2048 oyunu için kullanılabilecek yöntemler hakkında yapay zeka araştırmacıları arasında ortak bir karar alınamadı. Bunun sebebi araştırmacıların bir kısmı bunun bir öğrenme problemi (pekiştirmeli öğrenme, yapay sinir ağları, N-Tuple, vb) olduğunu savunmakta (Kondo and Matsuzaki, 2019;Boris and Goran, 2017) iken diğer araştırmacılar ise bunun bir arama problemi (Minimax, Expectimax, Averaged Depth Limited Search, Breath First Search, vb) olduğunu öne sürdüler (Nie et al, n.d.; Rodgers and Levine, n.d.). Bu alanda son çalışmalarda 2048 oyununun daha çok öğrenme problemi olarak değerlendirildiği görüldü.…”