Poker-CNN: A Pattern Learning Strategy for Making Draws and Bets in Poker Games Using Convolutional Networks

Abstract

Poker is a family of card games that includes many varia- tions. We hypothesize that most poker games can be solved as a pattern matching problem, and propose creating a strong poker playing system based on a unified poker representa- tion. Our poker player learns through iterative self-play, and improves its understanding of the game by training on the results of its previous actions without sophisticated domain knowledge. We evaluate our system on three poker games: single player video poker, two-player Limit Texas Hold’em, and finally two-player 2-7 triple draw poker. We show that our model can quickly learn patterns in these very different poker games while it improves from zero knowledge to a competi- tive player against human experts. The contributions of this paper include: (1) a novel represen- tation for poker games, extendable to different poker vari- ations, (2) a Convolutional Neural Network (CNN) based learning model that can effectively learn the patterns in three different games, and (3) a self-trained system that signif- icantly beats the heuristic-based program on which it is trained, and our system is competitive against human expert players.

AlphaHoldem: High-Performance Artificial Intelligence for Heads-Up No-Limit Poker via End-to-End Reinforcement Learning

“…For state representation comparison, we consider three alternative methods: 1) Vectorized state representation like DeepCFR (Brown et al 2019) (Vector). It uses vectors to represent the card information (two 52-dimensional vectors) and the action information (each betting position represented by a binary value specifying whether a bet has occurred and a float value specifying the bet size); 2) PokerCNN-based state representation (Yakovenko et al 2016) (PokerCNN) uses 3D tensors to represent card and action information together and use a single ConvNet to learn features; 3) State representation without history information (W/O History Information) is similar to AlphaHoldem except that it does not contain history action information.…”

“…Some recent works also make efforts towards this direction. NFSP (Heinrich and Silver 2016) and Poker-CNN (Yakovenko et al 2016) have approached state-of-the-art performance in limit Texas hold'em. DeepCFR (Brown et al 2019) further improves the performance by approximates CFR's behavior in the game using deep neural networks and Discounted CFR (Brown and Sandholm 2019a).…”

Supervised and Reinforcement Learning from Observations in Reconnaissance Blind Chess

“…In poker, some previous work exists on directly learning from observations. [12] learned to play simple poker versions from observations of hands, which resulted in a good, but not very strong performance. [13] proposed a self-play algorithm that guarantees to converge to a Nash equilibrium.…”

A Deep Reinforcement Learning Approach for Airport Departure Metering Under Spatial–Temporal Airside Interactions

“…2) Using Reinforcement Learning to solve MDPs with large state-action space: Reinforcement learning (RL) has been used to solve sequential decision-making problems modelled as MDPs. Notable achievements of RL include playing sequential decision-making games like chess [21] and poker [22] at super human level. To the best of our knowledge, only Mori [19] till date has investigated the potential of RL to devise optimal DM policy.…”

“…ICAO denes the engine's fuel ows at four different thrust levels: 100%, 85%, 30%, and 7% of engine maximum power corresponding to four different conditions: take-off, climb out, approach and idle, respectively. The thrust level ϵ is specied by the thrust at the time t and the engine's maximum thrust using equation (22).…”