Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

Liu, Bo; Liu, Qiang; Stone, Peter; Garg, Animesh; Zhu, Yuke; Anandkumar, Animashree

doi:10.48550/arxiv.2105.08692

Cited by 2 publications

(2 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In recent times, Majumdar et al [22] present a line of work where MADDPG agents learn different strategies by de-coupling and automatically weighting individual and global goals in a population-based training paradigm. Liu et al [18] consider the problem of coordination with dynamic composition, where a coach agent with global view distribute individual strategies to dynamically varying player agents. Jiang et al [12] promote sub-task specialization via emergence of individuality through generating dynamic intrinsic rewards.…”

Section: Related Literaturementioning

confidence: 99%

Opponent-aware Role-based Learning in Team Competitive Markov Games

Koley¹,

Maiti²,

Ganguly³

et al. 2023

Preprint

View full text Add to dashboard Cite

Team competition in multi-agent Markov games is an increasingly important setting for multi-agent reinforcement learning, due to it's general applicability in modeling many real-life situations. Multiagent actor-critic methods are the most suitable class of techniques for learning optimal policies in the team competition setting, due to their flexibility in learning agent-specific critic functions, which can also learn from other agents. In many real-world team competitive scenarios, the roles of the agents naturally emerge, in order to aid in coordination and collaboration within members of the teams. However, existing methods for learning emergent roles rely heavily on the Q-learning setup which does not allow learning of agent-specific Q-functions. In this paper, we propose RAC, a novel technique for learning the emergent roles of agents within a team that are diverse and dynamic. In the proposed method, agents also benefit from predicting the roles of the agents in the opponent team. RAC uses the actor-critic framework with role encoder and opponent role predictors for learning an optimal policy. Experimentation using 2 games demonstrates that the policies learned by RAC achieve higher rewards than those learned using state-of-the-art baselines. Moreover, experiments suggest that the agents in a team learn diverse and opponent-aware policies.

show abstract

Section: Related Literaturementioning

confidence: 99%

Opponent-aware Role-based Learning in Team Competitive Markov Games

Koley¹,

Maiti²,

Ganguly³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, [37] present an extension of MADDPG with separately learning individual and global goals in a populationbased training paradigm. [38] tackles the problem of dynamic team composition in coach-player paradigm. However, none of them explicitly address the setting of unequal competition, with focus on effects of incentives in offsetting the inequality.…”

Section: Related Workmentioning

confidence: 99%

Offsetting Unequal Competition through RL-assisted Incentive Schemes

Koley¹,

Maiti²,

Bhattacharya³

et al. 2022

Preprint

View full text Add to dashboard Cite

This paper investigates the dynamics of competition among organizations with unequal expertise. Multi-agent reinforcement learning has been used to simulate and understand the impact of various incentive schemes designed to offset such inequality. We design Touch-Mark, a game based on wellknown multi-agent-particle-environment, where two teams (weak, strong) with unequal but changing skill levels compete against each other. For training such a game, we propose a novel controller assisted multi-agent reinforcement learning algorithm C-MADDPG which empowers each agent with an ensemble of policies along with a supervised controller that by selectively partitioning the sample space, triggers intelligent role division among the teammates. Using C-MADDPG as an underlying framework, we propose an incentive scheme for the weak team such that the final rewards of both teams become the same. We find that in spite of the incentive, the final reward of the weak team falls short of the strong team. On inspecting, we realize that an overall incentive scheme for the weak team does not incentivize the weaker agents within that team to learn and improve. To offset this, we now specially incentivize the weaker player to learn and as a result, observe that the weak team beyond an initial phase performs at par with the stronger team. The final goal of the paper has been to formulate a dynamic incentive scheme that continuously balances the reward of the two teams. This is achieved by devising an incentive scheme enriched with an RL agent which takes minimum information from the environment.

show abstract

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

Cited by 2 publications

References 29 publications

Opponent-aware Role-based Learning in Team Competitive Markov Games

Opponent-aware Role-based Learning in Team Competitive Markov Games

Offsetting Unequal Competition through RL-assisted Incentive Schemes

Contact Info

Product

Resources

About