Emulating Human Play in a Leading Mobile Card Game

Abstract

Monte Carlo Tree Search (MCTS) has become a popular solution for game AI, capable of creating strong game playing opponents. However, the emergent playstyle of agents using MCTS is not necessarily human-like, believable or enjoyable. AI Factory Spades, currently the top rated Spades game in the Google Play store, uses a variant of MCTS to control AI allies and opponents. In collaboration with the developers, we showed in a previous study that the playstyle of human players significantly differed from that of the AI players [1]. This article presents a method for player modelling using gameplay data and neural networks that does not require domain knowledge, and a method of biasing MCTS with such a player model to create Spades playing agents that emulate human play whilst maintaining strong, competitive performance. The methods of player modelling and biasing MCTS presented in this study are applied to the commercial codebase of AI Factory Spades, and are transferable to MCTS implementations for discrete-action games where relevant gameplay data is available.

Policy Based Inference in Trick-Taking Card Games

“…In other domains, Richards and Amir [15] model the opponent's policy using a static evaluation technique and then perform inference on the opponent's remaining tiles given their most recent move in Scrabble. Baier et al [16] leverage policies trained from supervised human data to bias MCTS results; this is similar to our approach in that it uses human data to train an opponent model, but different because their model is not used to infer opponent hidden information. Sturtevant and Bowling [17] build a generalized model of the opponent from a set of candidate player strategies.…”

Bidding in Spades

“…Risk seeking bids when opponents are winning:. When the opponents can win the game on this round, BIS is modifying her bid in four cases: (1) opponents have bid nil, (2) high sum of bids, (3) opponents are winning by small points gap, or (4) opponents are winning by a medium points gap and BIS has a risky nil hand.…”

“…Two groups have made intensive research in the specific area of Spades agents: a group from University of Alberta [19,20,21,22] and AI-Factory group [2,8,9,25]. The latter launched a commercial application called Spades Free.…”

“…Whitehouse et al [25] presented an improved MCTS playing module that beats their strongest heuristic playing agent, they explain that the MCTS must use several heuristics tweaks to be perceived as strong by human players. In a follow-up study, Baier et al [2] used neural networks in order to emulate human play. They trained the network to predict human's next moves, given a game state.…”

Winning Isn't Everything: Enhancing Game Development with Intelligent Agents

“…MCTS has also been applied to the game of 7 Wonders [39] and Ticket to Ride [40]. Furthermore, Baier et al biased MCTS with a player model, extracted from game-play data, to have an agent that was competitive while approximating human-like play [41]. Tesauro [42], on the other hand, used TD-Lambda to train Backgammon agents at a superhuman level.…”