Learning enables adaptation in cooperation for multi-player stochastic games

Huang, Feng; Cao, Ming; Wang, Long

doi:10.1098/rsif.2020.0639

Cited by 13 publications

(9 citation statements)

References 74 publications

(138 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the state s 3 , an n-player snowdrift game (nSD) is played, and players' payoffs are calculated by a (s l | s 3 ) = ϑ s l − c/ and b (s l | s 3 ) = ϑ s l if > 0, and otherwise b (s l | s 3 ) = 0, where ϑ s l , as a function of the state s l at the next time, is the benefit of players when there exists at least one player choosing action C in the group. (see [45] and references therein for more details of these three games.) The simulation results are shown in Fig.…”

Section: B the Convergence Analysismentioning

confidence: 99%

See 1 more Smart Citation

Robust optimal policies for team Markov games

Huang¹,

Cao²,

Wang³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In stochastic dynamic environments, team stochastic games have emerged as a versatile paradigm for studying sequential decision-making problems of fully cooperative multiagent systems. However, the optimality of the derived policies is usually sensitive to the model parameters, which are typically unknown and required to be estimated from noisy data in practice. To mitigate the sensitivity of the optimal policy to these uncertain parameters, in this paper, we propose a model of "robust" team stochastic games, where players utilize a robust optimization approach to make decisions. This model extends team stochastic games to the scenario of incomplete information and meanwhile provides an alternative solution concept of robust team optimality. To seek such a solution, we develop a learning algorithm in the form of a Gauss-Seidel modified policy iteration and prove its convergence. This algorithm, compared with robust dynamic programming, not only possesses a faster convergence rate, but also allows for using approximation calculations to alleviate the curse of dimensionality. Moreover, some numerical simulations are presented to demonstrate the effectiveness of the algorithm by generalizing the game model of social dilemmas to sequential robust scenarios.

show abstract

Section: B the Convergence Analysismentioning

confidence: 99%

“…To demonstrate the effectiveness of GSraMPI numerically, we here generalize the stochastic game model of social dilemmas in [45] to the scenario of incomplete information, and we…”

Section: Theorem 4 For Any Nonnegative Integer Sequence {M T } T∈n I...mentioning

confidence: 99%

Robust optimal policies for team Markov games

Huang¹,

Cao²,

Wang³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Conversely, cooperation described by providing public goods is universal ranging from microorganism to human society [6][7][8][9][10][11]. Thus what drives the emergence of cooperative behaviour has attracted widespread attention from a broad range of disciplines [12][13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Early exclusion leads to cyclical cooperation in repeated group interactions

Liu

Xiao

Chen

et al. 2022

J. R. Soc. Interface.

View full text Add to dashboard Cite

Explaining the emergence and maintenance of cooperation among selfish individuals from an evolutionary perspective remains a grand challenge in biology, economy and social sciences. Social exclusion is believed to be an answer to this conundrum. However, previously related studies often assume one-shot interactions and ignore how free-riding is identified, which seem to be too idealistic. In this work, we consider repeated interactions where excluders need to pay a monitoring cost to identify free-riders for exclusion and free-riders cannot participate in the following possible game interactions once they are identified and excluded by excluders in the repeated interaction process. We reveal that the introduction of such exclusion can prevent the breakdown of cooperation in repeated group interactions. In particular, we demonstrate that an evolutionary oscillation among cooperators, defectors and excluders can appear in infinitely large populations when early exclusion is implemented. In addition, we find that the population spends most of the time in states where cooperators dominate for early exclusion when stochastic mutation–selection is considered in finite populations. Our results highlight that early exclusion is successful in solving the mentioned enigma of cooperation in repeated group interactions.

show abstract

“…Conversely, cooperation described by providing public goods is universal ranging from microorganism to human society [6][7][8][9][10][11]. Thus what drives the emergence of cooperative behaviour has attracted widespread attention from broad range of disciplines [12][13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Early exclusion leads to cyclical cooperation in repeated group interactions

Liu,

Xiao,

Chen

et al. 2022

Preprint

View full text Add to dashboard Cite

Explaining the emergence and maintenance of cooperation among selfish individuals from an evolutionary perspective remains a grand challenge in biology, economy, and social sciences. Social exclusion is believed to be an answer to this conundrum. However, previously related studies often assume one-shot interactions and ignore how free-riding is identified, which seem to be too idealistic. In this work, we consider repeated interactions where excluders need to pay a monitoring cost to identify free-riders for exclusion and free-riders cannot participate in the following possible game interactions once they are identified and excluded by excluders in the repeated interaction process. We reveal that the introduction of such exclusion can prevent the breakdown of cooperation in repeated group interactions.In particular, we demonstrate that an evolutionary oscillation among cooperators, defectors, and excluders can appear in infinitely large populations when early exclusion is implemented. In addition, we find that the population spends most of the time in states where cooperators dominate for early exclusion when stochastic mutation-selection is considered in finite populations. Our results highlight that early exclusion is successful in solving the mentioned enigma of cooperation in repeated group interactions.

show abstract

Learning enables adaptation in cooperation for multi-player stochastic games

Cited by 13 publications

References 74 publications

Robust optimal policies for team Markov games

Robust optimal policies for team Markov games

Early exclusion leads to cyclical cooperation in repeated group interactions

Early exclusion leads to cyclical cooperation in repeated group interactions

Contact Info

Product

Resources

About