Safe opponent exploitation

Abstract

We consider the problem of playing a finitely-repeated two-player zero-sum game safely-that is, guaranteeing at least the value of the game per period in expectation regardless of the strategy used by the opponent. Playing a stage-game equilibrium strategy at each time step clearly guarantees safety, and prior work has conjectured that it is impossible to simultaneously deviate from a stage-game equilibrium (in hope of exploiting a suboptimal opponent) and to guarantee safety. We show that such profitable deviations are indeed possible-specifically, in games where certain types of 'gift' strategies exist, which we define formally. We show that the set of strategies constituting such gifts can be strictly larger than the set of iteratively weakly-dominated strategies; this disproves another recent conjecture which states that all non-iterativelyweakly-dominated strategies are best responses to each equilibrium strategy of the other player. We present a full characterization of safe strategies, and develop efficient algorithms for exploiting suboptimal opponents while guaranteeing safety. We also provide analogous results for sequential perfect and imperfectinformation games, and present safe exploitation algorithms and full characterizations of safe strategies for those settings as well. We present experimental results in Kuhn poker, a canonical test problem for game-theoretic algorithms. Our experiments show that 1) aggressive safe exploitation strategies significantly outperform adjusting the exploitation within equilibrium strategies and 2) all the safe exploitation strategies significantly outperform a (non-safe) best response strategy against strong dynamic opponents.

Synergistic Information Processing Encrypts Strategic Reasoning in Poker

“…Despite major breakthroughs in artificial intelligence approaches to poker, there remains surprisingly little understanding of how human experts play, or of how human expertise might better inform algorithm design. Our results on the encrypting function of synergistic information may constitute behavioral evidence for the use of “safe” strategies, a type of non‐equilibrium but still non‐exploitable behavior that was recently introduced by computational game theorists to help algorithms respond more adaptively to irrational play (Ganzfried & Sandholm, ).…”

A Survey of Opponent Modeling in Adversarial Domains

“…It is not possible to exploit an opponent's strategy, by deviating from the equilibrium strategy, without creating the possibility of being exploited oneself. Although exceptions were later discovered (Ganzfried & Sandholm, 2015), this general tradeoff between exploitation and safety is sometimes called the "get-taught-and-exploited problem" (Sandholm, 2007), and has been applied in spirit to non-game-theoretic domains as well (Biswas et al, 2014). The concept of agent misdirection has also spawned interest in strategies for counter-misdirection (Chen & Arkin, 2021).…”

Subgame solving without common knowledge

“…In other words, every affine equilibrium is an NE of the restricted game Γ ′ in which ⊖ can only play her NE strategies in Γ. That is, affine equilibria are not exploitable by NE strategies of the opponent, not even by safe exploitation techniques [10]. So, the only way for the opponent to exploit an affine equilibrium is to open herself up to counter-exploitation.…”