SAI a Sensible Artificial Intelligence that plays Go

Abstract

We propose a multiple-komi modification of the AlphaGo Zero/Leela Zero paradigm. The winrate as a function of the komi is modeled with a two-parameters sigmoid function, hence the winrate for all komi values is obtained, at the price of predicting just one more variable. A second novel feature is that training is based on self-play games that occasionaly branch -with changed komi-when the position is uneven. With this setting, reinforcement learning is shown to work on 7×7 Go, obtaining very strong playing agents. As a useful byproduct, the sigmoid parameters given by the network allow to estimate the score difference on the board, and to evaluate how much the game is decided. Finally, we introduce a family of agents which target winning moves with a higher score difference.

Biasing MCTS with Features for General Games

“…In the majority of research based on deep learning for board games in more recent years, convolutional neural networks (CNNs) (LeCun et al, 1989) are used (Silver et al, 2016(Silver et al, , 2017Anthony et al, 2017;Lorentz & Zosa IV, 2017;Silver et al, 2018;Tian et al, 2019;Morandin et al, 2019;Wu, 2019;Cohen-Solal, 2020;Cazenave et al, 2020;Soemers et al, 2021a;Cohen-Solal & Cazenave, 2021;Soemers et al, 2021b). These typically operate on raw, low-level state inputs, as opposed to the higher-level features considered in this paper.…”

Manipulating the Distributions of Experience used for Self-Play Learning in Expert Iteration

“…Generating data (experience) as described above is the most common procedure, and has produced state-of-the-art results empirically [1], [3], but it is not certain that the optimal loss function is one that weights states s by p(s) as in Equation (9). It is possible that different weightings may perform better.…”

“…Even if we expect the cross-entropy loss function in Equation (9), where states s are weighted by p(s), to be the optimal one to optimise. It is still possible that approaches leading to experience buffers with different data distributions, or approaches that sample from it in a different (non-uniform) manner, may be expected to perform more successfully.…”

Derived metrics for the game of Go -- intrinsic network strength assessment and cheat-detection

“…Convolutional neural networks can have different heads, giving other values beyond a probability distribution for the next move. They can be trained to predict the score lead, the score difference at the end of the game [8,9,17]. The score value head combined with the Monte-Carlo search methods give statistical information about the outcome of the game: the scoremean value.…”

“…Several new implementations followed the success of AG and AGZ [3,8,9,16,17]. Given some computational resources, now anyone can build a deep learning Go engine [12].…”