Evolution of flexibility and rigidity in retaliatory punishment

Abstract

Natural selection designs some social behaviors to depend on flexible learning processes, whereas others are relatively rigid or reflexive. What determines the balance between these two approaches? We offer a detailed case study in the context of a two-player game with antisocial behavior and retaliatory punishment. We show that each player in this game-a "thief" and a "victim"-must balance two competing strategic interests. Flexibility is valuable because it allows adaptive differentiation in the face of diverse opponents. However, it is also risky because, in competitive games, it can produce systematically suboptimal behaviors. Using a combination of evolutionary analysis, reinforcement learning simulations, and behavioral experimentation, we show that the resolution to this tension-and the adaptation of social behavior in this game-hinges on the game's learning dynamics. Our findings clarify punishment's adaptive basis, offer a case study of the evolution of social preferences, and highlight an important connection between natural selection and learning in the resolution of social conflicts. punishment | evolution | reinforcement learning | game theory | commitment H uman social behavior is sometimes remarkably rigid, and other times remarkably flexible. A key challenge for evolutionary theory is to understand why. That is, when will natural selection favor "reflexive" social behaviors, and when will it instead favor more flexible processes that guide social decisionmaking by learning?We investigate a case study of this problem that illuminates some general principles of the evolution of social cognition. Specifically, we model the dynamic between antisocial behavior and retaliatory punishment in repeated relationships. Our goal is to understand when natural selection will favor flexibility (e.g., "try stealing and see if you can get away with it") versus rigidity ("punish thieves no matter what"). We approach this question through both a game-theoretic model of punishment and agentbased simulations that allow for the evolution of the rewards that guide learning. We demonstrate that the evolution of punishment depends on the learning dynamics of competing flexible agents, and that this interaction between learning and evolution can produce individuals with innate "social preferences," such as a taste for revenge (1-4). The Evolution of Retaliatory PunishmentIndividuals often punish those who harm them, even at a cost to themselves (5, 6). The adaptive rationale of this behavior seems clear in repeated or reputational interactions: Punishment promises a long-run gain by deterring social partners from doing future harm. This logic was classically formalized with a simple two-party repeated game (5) (Fig. 1A). On each round, a thief has the option to either steal from a victim (earning s and inflicting a cost −s) or do nothing. In response, the victim may either punish (paying a cost −c to inflict a cost −p) or do nothing. Formal analysis shows that "punish all theft/stop stealing from victims who punish" is ev...

Extortion, intuition, and the dark side of reciprocity

“…Consequently, he should act just as the evolutionary player does, maximizing his fitness by fully cooperating. In this case, the extortionate ZD player dominates her partner not because of a structural asymmetry inherent in the game, but because of an asymmetry in the "rigidity" or "flexibility" of their respective strategies (see also Morris, MacGlashan, Littman, & Cushman 2017).…”

“…Vignette studies find that punishment judgments mostly depend on simple retributive motives rather than a representation of its long-run advantages (Carlsmith, Darley, & Robinson, 2002). This conclusion is reinforced by studies of punishment in economic games: People punish norm-violating behavior even in one-shot games, when it cannot improve their direct payoffs (Balafoutas, Grechenig, & Nikiforakis, 2014;Fehr & Fischbacher, 2004), even when it is anonymous, and thus cannot benefit their reputation (Güth, Schmittberger, Schwarze, 1982;Jordan, McAuliffe, & Rand, 2015;Jordan & Rand, 2020;Kahneman, Knetsch, Thaler, 1986;Straub & Murnighan, 1995), and even when they have ample evidence that it will not change their social partner's behavior (Crockett, Özdemir, & Fehr, 2014;Morris, MacGlashan, Littman & Cushman, 2017). Punishment is often motivated by a specific emotion, anger (Nelissen & Zeelenberg, 2009;Seip, Van Dijk, & Rotteveel, 2014).…”

Payback without bookkeeping: The origins of revenge and retaliation

“…Thus, the most plausible evolutionary trajectory from the symmetric game to the asymmetric game is a reserve strategy that is not learned. Second, recent work also suggests that punishment is unlikely to be learned when it is costly and strategies that always punish (without learning) are more likely to evolve if stealing resources (similar to violating a convention) is rewarding (Morris, Macglashan, Littman, & Cushman, 2017). fixed, that population could have instead broken symmetry with the paradoxical convention. If it had, the reserve strategy would still be in play.…”

When do we punish people who don’t?

“…As humans, we often punish those who are antisocial: People who do not cooperate (Balliet & Van Lange, 2013;de Quervain et al, 2004;Gächter, Renner, & Sefton, 2008;Mathew & Boyd, 2011), who cause harm (Bone & Raihani, 2015;Buckholtz et al, 2008;Martin & Cushman, 2016b;McCullough, Kurzban, & Tabak, 2013;Morris, MacGlashan, Littman, & Cushman, 2017;Treadway et al, 2014) who behave unfairly (Cushman, Dreber, Wang, & Costa, 2009;FeldmanHall, Sokol-Hessner, Van Bavel, & Phelps, 2014;Henrich et al, 2010Henrich et al, , 2006Martin & Cushman, 2015;Mendoza, Lane, & Amodio, 2014) or who violate norms (Balafoutas & Nikiforakis, 2012;Carpenter & Matthews, 2009;Fehr & Fischbacher, 2004b). Such punishment can enforce prosociality and uphold norms, whether on behalf of ourselves or others (Balliet, Mulder, & Van Lange, 2011;Carpenter & Matthews, 2009;Fehr & Fischbacher, 2004b;Gaechter, 2014).…”