MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning

Malysheva, Aleksandra; Kudenko, Daniel; Shpilman, Aleksei

doi:10.1109/redundancy48165.2019.9003345

Cited by 19 publications

(10 citation statements)

References 5 publications

(9 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The representation of the cognition of agent i is denoted as cognition vector C i ; it is based on the states of all entities within its observation range. The common methods are to adopt directly multilayer perception (MLP) [9][10][11] or a graph convolution network (GCN) [15,29,30] to extract features of the observations as the cognition vector C i . However, the cognitive vectors extracted by these methods are essentially the result of single-valued mapping from vector to vector.…”

Section: Definitionmentioning

confidence: 99%

Multiagent Hierarchical Cognition Difference Policy for Multiagent Cooperation

Wang

Liu

et al. 2021

Algorithms

View full text Add to dashboard Cite

Multiagent cooperation is one of the most attractive research fields in multiagent systems. There are many attempts made by researchers in this field to promote cooperation behavior. However, several issues still exist, such as complex interactions among different groups of agents, redundant communication contents of irrelevant agents, which prevents the learning and convergence of agent cooperation behaviors. To address the limitations above, a novel method called multiagent hierarchical cognition difference policy (MA-HCDP) is proposed in this paper. It includes a hierarchical group network (HGN), a cognition difference network (CDN), and a soft communication network (SCN). HGN is designed to distinguish different underlying information of diverse groups’ observations (including friendly group, enemy group, and object group) and extract different high-dimensional state representations of different groups. CDN is designed based on a variational auto-encoder to allow each agent to choose its neighbors (communication targets) adaptively with its environment cognition difference. SCN is designed to handle the complex interactions among the agents with a soft attention mechanism. The results of simulations demonstrate the superior effectiveness of our method compared with existing methods.

show abstract

Section: Definitionmentioning

confidence: 99%

Multiagent Hierarchical Cognition Difference Policy for Multiagent Cooperation

Wang

Liu

et al. 2021

Algorithms

View full text Add to dashboard Cite

show abstract

“…Each hidden layer has 64 neurons, and ReLU is used as the activation function, which is defined as Equation (11).…”

Section: Multi-agent Interactive Networkmentioning

confidence: 99%

“…Accordingly, multi-agent intelligent learning has been intensively studied and conceived as deep learning develops. For instance, Malysheva A. et al [11] proposed a new MAGNet method for multi-agent reinforcement learning. Based on a weight agnostic neural networks (WANNs) methodology, an automated searching neural net architecture strategy was proposed that can perform various tasks such as identifying zero-day attacks [12].…”

Section: Introductionmentioning

confidence: 99%

An Interactive Self-Learning Game and Evolutionary Approach Based on Non-Cooperative Equilibrium

Zhao

Zhang

et al. 2021

Electronics

View full text Add to dashboard Cite

Most current studies on multi-agent evolution based on deep learning take a cooperative equilibrium strategy, while interactive self-learning is not always considered. An interactive self-learning game and evolution method based on non-cooperative equilibrium (ISGE-NCE) is proposed to take the benefits of both game theory and interactive learning for multi-agent confrontation evolution. A generative adversarial network (GAN) is designed combining with multi-agent interactive self-learning, and the non-cooperative equilibrium strategy is well adopted within the framework of interactive self-learning, aiming for high evolution efficiency and interest. For assessment, three typical multi-agent confrontation experiments are designed and conducted. The results show that, first, in terms of training speed, the ISGE-NCE produces a training convergence rate of at least 46.3% higher than that of the method without considering interactive self-learning. Second, the evolution rate of the interference and detection agents reaches 60% and 80%, respectively, after training by using our method. In the three different experiment scenarios, compared with the DDPG, our ISGE-NCE method improves the multi-agent evolution effectiveness by 43.4%, 50%, and 20%, respectively, with low training costs. The performances demonstrate the significant superiority of our ISGE-NCE method in swarm intelligence.

show abstract

“…Works that directly represent multi-agent systems as graphs of agents include DGN [Jiang et al 2020], MAGNet [Malysheva et al 2019], NerveNet [Wang et al 2018b] and ]. DGN [Jiang et al 2020] introduced the use of graph convolutional layers for inter-agent communication, as well techniques to stabilize training when using these convolutions in RL tasks.…”

Section: Related Workmentioning

confidence: 99%

“…Until now, new work has focused in the approximation of policies for homogeneous agents, i.e. agents that share the same action set and policy , Malysheva et al 2019, Jiang et al 2020, or in the specialization of agents for a limited number of simple actions [Wang et al 2018a]. However, no work has explicitly studied the potential of creating neural network architectures for environments with heterogeneous agents, capable of specializing the approximated policies according to an agent's class or role in the environment.…”

Section: Introductionmentioning

confidence: 99%

Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

Meneghetti

Bianchi

2020

Anais Do Encontro Nacional De Inteligência Artificial E Computacional (ENIAC 2020)

View full text Add to dashboard Cite

This work proposes a neural network architecture that learns policies for multiple agent classes in a heterogeneous multi-agent reinforcement setting. The proposed network uses directed labeled graph representations for states, encodes feature vectors of different sizes for different entity classes, uses relational graph convolution layers to model different communication channels between entity types and learns distinct policies for different agent classes, sharing parameters wherever possible. Results have shown that specializing the communication channels between entity classes is a promising step to achieve higher performance in environments composed of heterogeneous entities.

show abstract

MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning

Cited by 19 publications

References 5 publications

Multiagent Hierarchical Cognition Difference Policy for Multiagent Cooperation

Multiagent Hierarchical Cognition Difference Policy for Multiagent Cooperation

An Interactive Self-Learning Game and Evolutionary Approach Based on Non-Cooperative Equilibrium

Towards Heterogeneous Multi-Agent Reinforcement Learning with Graph Neural Networks

Contact Info

Product

Resources

About