The Art of War: Beyond Memory-one Strategies in Population Games

Abstract

We show that the history of play in a population game contains exploitable information that can be successfully used by sophisticated strategies to defeat memory-one opponents, including zero determinant strategies. The history allows a player to label opponents by their strategies, enabling a player to determine the population distribution and to act differentially based on the opponent’s strategy in each pairwise interaction. For the Prisoner’s Dilemma, these advantages lead to the natural formation of cooperative coalitions among similarly behaving players and eventually to unilateral defection against opposing player types. We show analytically and empirically that optimal play in population games depends strongly on the population distribution. For example, the optimal strategy for a minority player type against a resident TFT population is ALLC, while for a majority player type the optimal strategy versus TFT players is ALLD. Such behaviors are not accessible to memory-one strategies. Drawing inspiration from Sun Tzu’s the Art of War, we implemented a non-memory-one strategy for population games based on techniques from machine learning and statistical inference that can exploit the history of play in this manner. Via simulation we find that this strategy is essentially uninvadable and can successfully invade (significantly more likely than a neutral mutant) essentially all known memory-one strategies for the Prisoner’s Dilemma, including ALLC (always cooperate), ALLD (always defect), tit-for-tat (TFT), win-stay-lose-shift (WSLS), and zero determinant (ZD) strategies, including extortionate and generous strategies.

Small groups and long memories promote cooperation

“…On the one hand, memory can obviously be a powerful force for promoting cooperation if it allows players to recognize kin, or to otherwise tag different opponents2627. Yet, such recognition is likely to be costly and complex to evolve232829.…”

An Open Framework for the Reproducible Study of the Iterated Prisoner’s Dilemma

“…This aspect of incomplete information is also considered in [6,24,33] where "noisy" tournaments randomly flip the choice made by a given strategy. In [34], incomplete information is considered in the sense of a probabilistic termination of each round of the tournament.…”

“…As mentioned before, IPD tournaments have been studied in an evolutionary con text: [12,24,36,42] consider this in a traditional evolutionary game theory context. These works investigate particular evolutionary contexts within which cooperation can evolve and persist.…”

“…This can be in the context of direct interactions between strategies or population dynamics for populations of many players using a variety of strategies, which can lead to very different results. For example, in [24] a machine learning algorithm in a population context outperforms strategies described in [36] and [42] that are claimed to dominate any evolutionary opponent in head-to-head interactions.…”

“…• [2,3] which describe a computer tournament that has been a major influence on sub sequent game theoretic work [5,6,7,8,10,11,12,13,15,18,23,24,27,32,33,34,36,41,42]. As described in [6] this work has not only had impact in mathemat ics but has also led to insights in biology (for example in [41], a real tournament where Blue Jays are the participants is described) and in particular in the study of evolution.…”