CadiaPlayer: Search-Control Techniques

Abstract

Effective search control is one of the key components of any successful simulation-based game-playing program. In General Game Playing (GGP), learning of useful search-control knowledge is a particularly challenging task because it must be done in real-time during online play. In here we describe the search-control techniques used in the 2010 version of the GGP agent CADIAPLAYER, and show how they have evolved over the years to become increasingly effective and robust across a wide range of games. In particular, we present a new combined search-control scheme (RAVE/MAST/FAST) for biasing action selection. The scheme proves quite effective on a wide range of games including chess-like games, which have up until now proved quite challenging for simulation-based GGP agents.

Playout policy adaptation with move features

“…FAST learns features such as piece values using TD(λ) [16]. FAST is used to bias playouts in combination with MAST but only slightly improves on MAST.…”

Recent Advances in General Game Playing

“…Most of the original contributions to the MCTS can be divided into three categories based on the area where they are applied: selection step, simulation step, or both. The last category was investigated by [ 20 ] by the authors of CadiaPlayer.…”

“…The RAVE method stands for Rapid Value Action Estimation. It was first proposed in 2007 for Go, but it was included in the CadiaPlayer's authors paper [ 20 ] for comparison and synergy purposes when it is be combined with other methods in GGP. A recent paper about RAVE which the GGP agents stem from is [ 21 ].…”

“…Four enhancements under the category of “simulation control” are investigated in the article [ 20 ] and the PhD thesis of one of the authors in [ 19 ].…”

“…This enhancement is called History Heuristic. In [ 20 ], historically good actions bias future simulations according to the Gibbs Sampling (or Boltzmann distribution). Tree-Only MAST (TO-MAST) : the same as MAST, but updates statistics only for actions within the constructed part of the UCT tree.…”