Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Yang, Zhengzhi; Zheng, Lei; Perc, Matjaž; Li, Yumeng

doi:10.1016/j.amc.2023.128364

Cited by 10 publications

(2 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For instance, Ding et al [62] found that individuals maintain a high level of cooperation act like WSLS in two-agent repeated games. Additionally, Zheng et al [63] observed elevated levels of trust in trust games, and Yang et al [64] proposed a reward information-sharing mechanism, indicating that considering neighbors' payoff information can significantly enhance cooperation.…”

Section: Introductionmentioning

confidence: 99%

The emergence of cooperation via Q-learning in spatial donation game

Zhang,

Rong,

Zheng

et al. 2024

J. Phys. Complex.

View full text Add to dashboard Cite

Decision-making often overlooks the feedback between agents and the environment. Reinforcement learning is widely employed through exploratory experimentation to address problems related to states, actions, rewards, decision-making in various contexts. This work considers a new perspective, where individuals continually update their policies based on interactions with the spatial environment, aiming to maximize cumulative rewards and learn the optimal strategy. Specifically, we utilize the Q-learning algorithm to study the emergence of cooperation in a spatial population playing the donation game. Each individual has a Q-table that guides their decision-making in the game. Interestingly, we find that cooperation emerges within this introspective learning framework, and a smaller learning rate and higher discount factor make cooperation more likely to occur. Through the analysis of Q-table evolution, we disclose the underlying mechanism for cooperation, which may provide some insights to the emergence of cooperation in the real-world systems.

show abstract

Section: Introductionmentioning

confidence: 99%

The emergence of cooperation via Q-learning in spatial donation game

Zhang,

Rong,

Zheng

et al. 2024

J. Phys. Complex.

View full text Add to dashboard Cite

show abstract

“…In view of the above, it is quite natural that one takes up the endeavour of not only considering different strategy update rules [49][50][51][52][53][54][55], that are not just dependent on the payoff [51], but also allow for the possibility that individuals may learn the appropriate rule (model for strategy update) from the experience gained through repeated interactions. In other words, a more complex strategy could involve adopting a strategy contingent on the belief about environmental state of player.…”

Section: Introductionmentioning

confidence: 99%

Inferring to cooperate: Evolutionary games with Bayesian inferential strategies

Patra,

Sengupta,

Paul

et al. 2024

New J. Phys.

View full text Add to dashboard Cite

Strategies for sustaining cooperation and preventing exploitation by selﬁsh agents in repeated games have mostly been restricted to Markovian strategies where the response of an agent depends on the actions in the previous round. Such strategies are characterized by lack of learning. However, learning from accumulated evidence over time and using the evidence to dynamically update our response is a key feature of living organisms. Bayesian inference provides a framework for such evidence-based learning mechanisms. It is therefore imperative to understand how strategies based on Bayesian learning fare in repeated games with Markovian strategies. Here, we consider a scenario where the Bayesian player uses the accumulated evidence of the opponent’s actions over several rounds to continuously update her belief about the reactive opponent’s strategy. The Bayesian player can then act on her inferred belief in diﬀerent ways. By studying repeated Prisoner’s dilemma games with such Bayesian inferential strategies, both in inﬁnite and ﬁnite populations, we identify the conditions under which such strategies can be evolutionarily stable. We ﬁnd that a Bayesian strategy that is less altruistic than the inferred belief about the opponent’s strategy can outperform a larger set of reactive strategies, whereas one that is more generous than the inferred belief is more successful when the beneﬁt-to-cost ratio of mutual cooperation is high. Our analysis reveals how learning the opponent’s strategy through Bayesian inference, as opposed to utility maximization, can be beneﬁcial in the long run, in preventing exploitation and eventual invasion by reactive strategies.

show abstract

The effect of intraspecific cooperation in a three-species cyclic predator-prey model

Dai,

Wang,

et al. 2024

Applied Mathematics and Computation

View full text Add to dashboard Cite

Interaction state Q-learning promotes cooperation in the spatial prisoner's dilemma game

Cited by 10 publications

References 55 publications

The emergence of cooperation via Q-learning in spatial donation game

The emergence of cooperation via Q-learning in spatial donation game

Inferring to cooperate: Evolutionary games with Bayesian inferential strategies

The effect of intraspecific cooperation in a three-species cyclic predator-prey model

Contact Info

Product

Resources

About