Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

Mao, Hangyu; Liu, Wulong; Hao, Jianye; Luo, Jun; Li, Dong; Zhang, Zhengchao; Wang, Jun; Xiao, Zhen

doi:10.1609/aaai.v34i05.6212

Cited by 52 publications

(21 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, in the direct communication method of embedding communication channels in deep neural networks, the use of specific communication channels can selectively complete the information exchange between various agents [20]. For learning communication protocols, it has been proven to be very effective [21][22][23]. Under normal circumstances, the continuous transmission between agents through the network forms a communication channel, which makes the agents consider local information and global information at the same time during the learning process.…”

Section: Related Workmentioning

confidence: 99%

Cooperative Multiagent Attentional Communication for Large-Scale Task Space

Zou

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

With the rapid development of mobile robots, they have begun to be widely used in industrial manufacturing, logistics scheduling, intelligent medical, and other fields. For large-scale task space, the communication between multiagents is the key to affect cooperation productivity, and agents can coordinate more effectively with the help of dynamic communication. However, the traditional communication mechanism uses simple message aggregation and broadcast and, in some cases, lacks the distinction of the importance of information. Multiagent deep reinforcement learning (MDRL) is valid to solve the problem of informational coordination strategies. However, how different messages affect each agent’s decision-making process remains a challenging task for large-scale task. To solve this problem, we propose IMANet (Import Message Attention Network). It divides the decision-making process into two substages: communication and action, where communication is considered to be part of the environment. First, an attention mechanism based on query vectors is introduced. The correlation between the query vector agent’s own information and the current state information of other agents is estimated, and then, the results are used to distinguish the importance of information from other agents. Second, the LSTM network is used as the unit controller for each agent, and individual rewards are used to guide the agent training after communication. Finally, IMANet is evaluated on tasks on challenging multi-agent platforms, Predator and Prey (PP), and traffic junction. The results show that IMANet can improve the efficiency of learning and training, especially when applied to large-scale task space, with a success rate 12% higher than CommNet in baseline experiments.

show abstract

Section: Related Workmentioning

confidence: 99%

Cooperative Multiagent Attentional Communication for Large-Scale Task Space

Zou

et al. 2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

show abstract

“…Weighted QMIX [18] proposes weighted projection to decompose any joint action-value functions. There are other works that investigate into MARL from the perspective of coordination graphs [24][25][26], communication [27,28,10], and role-based learning [12,29].…”

Section: Related Workmentioning

confidence: 99%

“…SEAC [35] partially solves this problem by sharing trajectories only for off-policy training. NCC [28] maintains cognition consistency by representation alignment between neighbors. Roy et al [36] force each agent to predict others' local policies and adds a coach for group experience alignment.…”

Section: Related Workmentioning

confidence: 99%

Celebrating Diversity in Shared Multi-Agent Reinforcement Learning

Li¹,

Wu²,

Wang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, deep multi-agent reinforcement learning (MARL) has shown the promise to solve complex cooperative tasks. Its success is partly because of parameter sharing among agents. However, such sharing may lead agents to behave similarly and limit their coordination capacity. In this paper, we aim to introduce diversity in both optimization and representation of shared multi-agent reinforcement learning. Specifically, we propose an information-theoretical regularization to maximize the mutual information between agents' identities and their trajectories, encouraging extensive exploration and diverse individualized behaviors. In representation, we incorporate agent-specific modules in the shared neural network architecture, which are regularized by L1-norm to promote learning sharing among agents while keeping necessary diversity. Empirical results show that our method achieves stateof-the-art performance on Google Research Football 2 and super hard StarCraft II micromanagement tasks. * Denotes equal contribution.2 Videos available at https://sites.google.com/view/celebrate-diversity-shared Preprint. Under review.

show abstract

“…These methods usually bring about some interdisciplinary tricks into MDRL field to help solve MAS problems. Q-value path decomposition (QDP) [27] integrates gradient attribution technique into MDRL to directly decompose global Q-values along trajectory paths to assign credits for agents, it has also been applied in StarCraft II micromanagement tasks with good performance; NCC-MARL [28] introduces neighborhood cognitive consistency (NCC) into MDRL to facilitate large-scale teamwork tasks like football player control. Mao et al [29] propose a novel reward design method to accelerate the formation of better policies, the method is specially designed for packet routing application.…”

Section: Related Workmentioning

confidence: 99%

A Novel Multi-Agent Parallel-Critic Network Architecture for Cooperative-Competitive Reinforcement Learning

et al. 2020

View full text Add to dashboard Cite

multi-agent deep reinforcement learning (MDRL) is an emerging research hotspot and application direction in the field of machine learning and artificial intelligence. MDRL covers many algorithms, rules and frameworks, it is currently researched in swarm system, energy allocation optimization, stocking analysis, sequential social dilemma, and with extremely bright future. In this paper, a parallel-critic method based on classic MDRL algorithm MADDPG is proposed to alleviate the training instability problem in cooperative-competitive multi-agent environment. Furthermore, a policy smoothing technique is introduced to our proposed method to decrease the variance of learning policies. The suggested method is evaluated in three different scenarios of authoritative multi-agent particle environment (MPE). Multiple statistical data of experimental results show that our method significantly improves the training stability and performance compared to vanilla MADDPG.INDEX TERMS multi-agent system, deep reinforcement learning, parallel-critic architecture, training stability.

show abstract

Neighborhood Cognition Consistent Multi-Agent Reinforcement Learning

Cited by 52 publications

References 10 publications

Cooperative Multiagent Attentional Communication for Large-Scale Task Space

Cooperative Multiagent Attentional Communication for Large-Scale Task Space

Celebrating Diversity in Shared Multi-Agent Reinforcement Learning

A Novel Multi-Agent Parallel-Critic Network Architecture for Cooperative-Competitive Reinforcement Learning

Contact Info

Product

Resources

About