Robust program equilibrium

Abstract

One approach to achieving cooperation in the one-shot prisoner's dilemma is Tennenholtz's (Games Econ Behav 49(2):363-373, 2004) program equilibrium, in which the players of a game submit programs instead of strategies. These programs are then allowed to read each other's source code to decide which action to take. As shown by Tennenholtz, cooperation is played in an equilibrium of this alternative game. In particular, he proposes that the two players submit the same version of the following program: cooperate if the opponent is an exact copy of this program and defect otherwise. Neither of the two players can benefit from submitting a different program. Unfortunately, this equilibrium is fragile and unlikely to be realized in practice. We thus propose a new, simple program to achieve more robust cooperative program equilibria: cooperate with some small probability and otherwise act as the opponent acts against this program. I argue that this program is similar to the tit for tat strategy for the iterated prisoner's dilemma. Both "start" by cooperating and copy their opponent's behavior from "the last round". We then generalize this approach of turning strategies for the repeated version of a game into programs for the one-shot version of a game to other two-player games. We prove that the resulting programs inherit properties of the underlying strategy. This enables them to robustly and effectively elicit the same responses as the underlying strategy for the repeated game.

Designing Preferences, Beliefs, and Identities for Artificial Intelligence

“…This turns the problem into a social choice problem; the link to social choice in such a context has already been observed several times (Greene et al 2016;Conitzer et al 2017;Noothigattu et al 2018;Zhang and Conitzer 2019). This general approach has already been used in the context of algorithms for finding matchings in kidney exchanges (Freedman et al 2018) as well as in the context of self-driving cars making emergency decisions (Noothigattu et al 2018). What is the best way to aggregate multiple objective functions into a single one?…”

“…If so, we may wish to sample some stakeholders, elicit what each of them thinks should be the objective function according to which the agent operates, and then aggregate these multiple objective functions into a single consistent objective function for the agent. This turns the problem into a social choice problem; the link to social choice in such a context has already been observed several times (Greene et al 2016;Conitzer et al 2017;Noothigattu et al 2018;Zhang and Conitzer 2019). This general approach has already been used in the context of algorithms for finding matchings in kidney exchanges (Freedman et al 2018) as well as in the context of self-driving cars making emergency decisions (Noothigattu et al 2018).…”

“…We can learn a model that predicts how someone would rank any pair of alternatives; see, e.g.,Noothigattu et al (2018). Still, such a model must be more concise than an arbitrary ranking.…”

Game-Theoretic Models of Moral and Other-Regarding Agents (extended abstract)

“…The other strand of ideas relevant to our work concerns the concept of program equilibria, defined in [66] and further investigated in [24,41,37,44,6,19,48]. There are several other related (and relevant) models, such as the translucent player model of [15,31], or mediated equilibria [46].…”

“…Among them: -fragility: (Kantian) program equilibria are sensitive (see e.g. [48]) to the precise specification of programs: do we insist that all agent programs are syntactically identical, or just "do the same thing"? See [44,37] for some attempted solutions for program equilibria that could be adapted to our setting.…”