Introspection dynamics: a simple model of counterfactual learning in asymmetric games

Abstract

Social behavior in human and animal populations can be studied as an evolutionary process. Individuals often make decisions between different strategies, and those strategies that yield a fitness advantage tend to spread. Traditionally, much work in evolutionary game theory considers symmetric games: Individuals are assumed to have access to the same set of strategies, and they experience the same payoff consequences. As a result, they can learn more profitable strategies by imitation. However, interactions are oftentimes asymmetric. In that case, imitation may be infeasible (because individuals differ in the strategies they are able to use), or it may be undesirable (because individuals differ in their incentives to use a strategy). Here, we consider an alternative learning process which applies to arbitrary asymmetric games, \textit{introspection dynamics}. According to this dynamics, individuals regularly compare their present strategy to a randomly chosen alternative strategy. If the alternative strategy yields a payoff advantage, it is more likely adopted. In this work, we formalize introspection dynamics for pairwise games. We derive simple and explicit formulas for the abundance of each strategy over time and apply these results to several well-known social dilemmas. In particular, for the volunteer's timing dilemma, we show that the player with the lowest cooperation cost learns to cooperate without delay.

Diversity enables the jump towards cooperation for the Traveler’s Dilemma

“…The stationary distribution of actions, derived using the main analytical result of Couto, Giaimo & Hilbe. (2022) 22 , allows the calculation of the average claim for different reward parameter (R) and selection intensity coefficient (β ) values. As shown in figure 4.c, strong selection and a low reward value are requirements for the convergence to high claims.…”

“…Finally, we should note that, in contrast to our findings for the Traveler's Dilemma, cooperation does not emerge in the Prisoner's Dilemma when played using introspection dynamics. There, both players learn to defect at all times, that is, play Nash, in the limit of strong selection 22 . Therefore, diversity is only effective on promoting cooperation when the structure of the game allows it.…”

“…For weak selection (β → 0), payoffs are not important in the learning process as any alternative action is randomly adopted. For strong selection (β → ∞), players exclusively adopt the alternative action if their payoff matches at least the realised payoff 22 . Therefore, β determines the magnitude of the impact that payoffs have on the game dynamics.…”

Diversity enables the jump towards cooperation for the Traveler’s Dilemma

“…Introspection dynamics is a learning model used to describe adaptive behaviour via introvert reasoning 21 , 22 . The basic idea is that a player, after considering her payoff in an encounter with an opponent, checks whether a randomly chosen alternative action—sampled from the uniform distribution on the [ L , U ] interval—would have given her a better payoff against a given action chosen by her opponent.…”

When costly migration helps to improve cooperation

“…In 2019, for instance, at least 3.5 % of the world's population comprised international migrants, according to the UN [1]. Individual mobility, however, is not just a simple reaction to a specific living condition; it could * szolnoki.attila@ek-cer.hu also be the source of a co-evolutionary mechanism of how collective well-being evolves in the framework of human cooperation [2][3][4][5][6][7][8][9]. Therefore, when we sought to understand the evolution of cooperation in spatially structured populations, considering mobile agents was among our core ideas from the outset [10][11][12].…”