Efficient Multi-Agent Communication via Shapley Message Value

Xue, Desheng; Yuan, Lei; Zhang, Zongzhang; Yu, Yang

doi:10.24963/ijcai.2022/82

Cited by 6 publications

(6 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We subjected T2MAC to rigorous testing across various MARL environments, such as Hallway, MPE, and SMAC. Compared to prominent multi-agent communication strategies like TarMac (Das et al 2019), MAIC (Yuan et al 2022), SMS (Xue et al 2022), and MASIA (Guan et al 2022), T2MAC consistently excelled in both performance and efficiency. Additionally, its versatility shone through across diverse scenarios.…”

Section: Introductionmentioning

confidence: 95%

“…Advancing this idea, (Kim et al 2019;Mao et al 2019;Wang et al 2020a;Sun et al 2021) implement a weight-based scheduler, prioritizing agents holding vital observations. Enriching this approach, I2C (Ding, Huang, and Lu 2020), MAGIC (Niu, Paleja, and Gombolay 2021), and SMS (Xue et al 2022) harness methods like causal inference, graph-attention, and Shapley message value to pinpoint ideal communication recipients.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration

Sun,

Zang,

et al. 2024

AAAI

View full text Add to dashboard Cite

Communication stands as a potent mechanism to harmonize the behaviors of multiple agents. However, existing work primarily concentrates on broadcast communication, which not only lacks practicality, but also leads to information redundancy. This surplus, one-fits-all information could adversely impact the communication efficiency. Furthermore, existing works often resort to basic mechanisms to integrate observed and received information, impairing the learning process. To tackle these difficulties, we propose Targeted and Trusted Multi-Agent Communication (T2MAC), a straightforward yet effective method that enables agents to learn selective engagement and evidence-driven integration. With T2MAC, agents have the capability to craft individualized messages, pinpoint ideal communication windows, and engage with reliable partners, thereby refining communication efficiency. Following the reception of messages, the agents integrate information observed and received from different sources at an evidence level. This process enables agents to collectively use evidence garnered from multiple perspectives, fostering trusted and cooperative behaviors. We evaluate our method on a diverse set of cooperative multi-agent tasks, with varying difficulties, involving different scales and ranging from Hallway, MPE to SMAC. The experiments indicate that the proposed model not only surpasses the state-of-the-art methods in terms of cooperative performance and communication efficiency, but also exhibits impressive generalization.

show abstract

Section: Introductionmentioning

confidence: 95%

Section: Related Workmentioning

confidence: 99%

T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration

Sun,

Zang,

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Communication plays a promising role in multi-agent coordination under partial observability [13,84]. Extensive research have been made on learning communication protocols to improve performance on cooperative tasks [20,15,34,73,12,37,77,71,21]. Previous works can be divided into two categories.…”

Section: Related Workmentioning

confidence: 99%

Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learning

Cong¹,

Chen²,

Liu³

et al. 2023

Preprint

View full text Add to dashboard Cite

Utilizing messages from teammates can improve coordination in cooperative Multi-agent Reinforcement Learning (MARL). Previous works typically combine raw messages of teammates with local information as inputs for policy. However, neglecting message aggregation poses significant inefficiency for policy learning. Motivated by recent advances in representation learning, we argue that efficient message aggregation is essential for good coordination in cooperative MARL. In this paper, we propose Multi-Agent communication via Self-supervised Information Aggregation (MASIA), where agents can aggregate the received messages into compact representations with high relevance to augment the local policy. Specifically, we design a permutation invariant message encoder to generate common information-aggregated representation from messages and optimize it via reconstructing and shooting future information in a self-supervised manner. Hence, each agent would utilize the most relevant parts of the aggregated representation for decisionmaking by a novel message extraction mechanism. Furthermore, considering the potential of offline learning for real-world applications, we build offline benchmarks for multi-agent communication, which is the first as we know. Empirical results demonstrate the superiority of our method in both online and offline settings. We also release the built offline benchmarks in this paper as a testbed for communication ability validation to facilitate further future research.

show abstract

“…Robustness in cooperative MARL Previous cooperative MARL [32] works either concentrate on improving coordination ability from diverse aspects like scalability [33], credit assignment [34], and non-stationarity [35], or applying the cooperative MARL technique to multiple domains like autonomous vehicle teams [36,37], power management [38], and dynamic algorithm configuration [39]. Those approaches ignore the robustness of the learned policy when encountering uncertainties, perturbations, or structural changes in the environment [10], hastening the robustness test in the MARL [40].…”

Section: Related Workmentioning

confidence: 99%

“…We design two instances, which are respectively GP-4r and GP-9r. In GP-4r, there are 4 (2×2) regions, of which each region is a field with size of [3,3], and three agents existing in the map, while, in GP-9r, there exist 9 (3×3) regions and 3 agents.…”

Section: • Gold Panner (Gp)mentioning

confidence: 99%

Communication-robust multi-agent learning by adaptable auxiliary multi-agent adversary generation

Yuan,

Chen,

Zhang

et al. 2023

Front. Comput. Sci.

Self Cite

View full text Add to dashboard Cite

Communication can promote coordination in cooperative Multi-Agent Reinforcement Learning (MARL). Nowadays, existing works mainly focus on improving the communication efficiency of agents, neglecting that real-world communication is much more challenging as there may exist noise or potential attackers. Thus the robustness of the communication-based policies becomes an emergent and severe issue that needs more exploration. In this paper, we posit that the ego system1) trained with auxiliary adversaries may handle this limitation and propose an adaptable method of Multi-Agent Auxiliary Adversaries Generation for robust Communication, dubbed MA3C, to obtain a robust communication-based policy. In specific, we introduce a novel message-attacking approach that models the learning of the auxiliary attacker as a cooperative problem under a shared goal to minimize the coordination ability of the ego system, with which every information channel may suffer from distinct message attacks. Furthermore, as naive adversarial training may impede the generalization ability of the ego system, we design an attacker population generation approach based on evolutionary learning. Finally, the ego system is paired with an attacker population and then alternatively trained against the continuously evolving attackers to improve its robustness, meaning that both the ego system and the attackers are adaptable. Extensive experiments on multiple benchmarks indicate that our proposed MA3C provides comparable or better robustness and generalization ability than other baselines.

show abstract

Efficient Multi-Agent Communication via Shapley Message Value

Cited by 6 publications

References 0 publications

T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration

T2MAC: Targeted and Trusted Multi-Agent Communication through Selective Engagement and Evidence-Driven Integration

Efficient Communication via Self-supervised Information Aggregation for Online and Offline Multi-agent Reinforcement Learning

Communication-robust multi-agent learning by adaptable auxiliary multi-agent adversary generation

Contact Info

Product

Resources

About