Policy Based Inference in Trick-Taking Card Games

Abstract

Trick-taking card games feature a large amount of private information that slowly gets revealed through a long sequence of actions. This makes the number of histories exponentially large in the action sequence length, as well as creating extremely large information sets. As a result, these games become too large to solve. To deal with these issues many algorithms employ inference, the estimation of the probability of states within an information set. In this paper, we demonstrate a Policy Based Inference (PI) algorithm that uses player modelling to infer the probability we are in a given state. We perform experiments in the German trick-taking card game Skat, in which we show that this method vastly improves the inference as compared to previous work, and increases the performance of the state-of-the-art Skat AI system Kermit when it is employed into its determinized search algorithm.

Opponent-Model Search in Games with Incomplete Information

“…Sturtevant, Zinkevich, and Bowling (2006) propose opponent models given by opponents' preferences over the outcomes of a game. Rebstock et al (2019) use opponent models learnt from human games for imperfect information (card) games. A survey of opponent modelling approaches is also provided by Albrecht and Stone (2018).…”

On the Power of Refined Skat Selection

“…With these evaluation functions, the bidding system selects the game with the best payoff. There are other machine learning efforts to predict bidding games and hand cards in skat [9], [23], [24], [25], [26]. Additionally, we have seen feature extraction in the related game of Hearts [27], and automated bidding improvements in the game of Spades [28].…”

“…Basing the strength of hands on winning probability or expected payoff rather than expected card points is more suitable, because the declarers' payoff in skat mostly depends on winning the game. Considering the bidding phase of [25], deep neural nets (DNN) were trained using recorded human data from a server play. Separate networks were trained for each game type except for null and null ouvert.…”

Knowledge-Based Paranoia Search in Trick-Taking

“…There have been larger efforts to apply machine learning to predict bidding options and hand cards in Skat [23], [22], [24], [25], [26], [10]. Additionally, we have seen feature extraction in the related game of Hearts [27], and automated bidding improvements in the game of Spades [28].…”