Multi-Agent Pattern Formation: a Distributed Model-Free Deep Reinforcement Learning Approach

Diallo, Elhadji Amadou Oury; Sugawara, Toshiharu

doi:10.1109/ijcnn48605.2020.9207657

“…So far, there has been little research on the RL algorithms of LMAS. The related studies focus on oversimplified tasks such as gridworld dotfighting and geometrical pattern-forming (Zheng et al 2018;Diallo and Sugawara 2020;Rubenstein, Cornejo, and Nagpal 2014).…”

Section: Introductionmentioning

confidence: 99%

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

Qingxu

¹

,

Qiu

²

,

Yi

³

et al. 2022

AAAI

View full text Add to dashboard Cite

When dealing with a series of imminent issues, humans can naturally concentrate on a subset of these concerning issues by prioritizing them according to their contributions to motivational indices, e.g., the probability of winning a game. This idea of concentration offers insights into reinforcement learning of sophisticated Large-scale Multi-Agent Systems (LMAS) participated by hundreds of agents. In such an LMAS, each agent receives a long series of entity observations at each step, which can overwhelm existing aggregation networks such as graph attention networks and cause inefficiency. In this paper, we propose a concentration network called ConcNet. First, ConcNet scores the observed entities considering several motivational indices, e.g., expected survival time and state value of the agents, and then ranks, prunes, and aggregates the encodings of observed entities to extract features. Second, distinct from the well-known attention mechanism, ConcNet has a unique motivational subnetwork to explicitly consider the motivational indices when scoring the observed entities. Furthermore, we present a concentration policy gradient architecture that can learn effective policies in LMAS from scratch. Extensive experiments demonstrate that the presented architecture has excellent scalability and flexibility, and significantly outperforms existing methods on LMAS benchmarks.

show abstract

“…So far, there has been little research on the RL algorithms of LMAS. The related studies focus on oversimplified tasks such as gridworld dotfighting and geometrical pattern-forming (Zheng et al 2018;Diallo and Sugawara 2020;Rubenstein, Cornejo, and Nagpal 2014).…”

Section: Introductionmentioning

confidence: 99%

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

Qingxu¹,

Qiu²,

Yi³

et al. 2022

Preprint

0

View full text Add to dashboard Cite

When dealing with a series of imminent issues, humans can naturally concentrate on a subset of these concerning issues by prioritizing them according to their contributions to motivational indices, e.g., the probability of winning a game. This idea of concentration offers insights into reinforcement learning of sophisticated Large-scale Multi-Agent Systems (LMAS) participated by hundreds of agents. In such an LMAS, each agent receives a long series of entity observations at each step, which can overwhelm existing aggregation networks such as graph attention networks and cause inefficiency. In this paper, we propose a concentration network called ConcNet. First, ConcNet scores the observed entities considering several motivational indices, e.g., expected survival time and state value of the agents, and then ranks, prunes, and aggregates the encodings of observed entities to extract features. Second, distinct from the well-known attention mechanism, ConcNet has a unique motivational subnetwork to explicitly consider the motivational indices when scoring the observed entities. Furthermore, we present a concentration policy gradient architecture that can learn effective policies in LMAS from scratch. Extensive experiments demonstrate that the presented architecture has excellent scalability and flexibility, and significantly outperforms existing methods on LMAS benchmarks.

show abstract

“…Çoklu robot sistemleri konusunda yapılmış çalışmalardan birisi olan [9]'daki çalışmada DQN algoritması kullanılarak, ortak bir deneyim tekrarı hafızası oluşturulmuş ve her ajan için ayrı bir yapay sinir ağı oluşturulmuştur. Her ajan için ayrı oluşturulan yapay sinir ağlarının belirlenen adım sayısına ulaşınca hedef ağ tarafından ağırlıkları eşitlenmektedir.…”

Section: Introductionunclassified

“…Her ajan için ayrı oluşturulan yapay sinir ağlarının belirlenen adım sayısına ulaşınca hedef ağ tarafından ağırlıkları eşitlenmektedir. [9]'daki sistemin blok diyagramı Şekil 1'de verilmiştir. [4]'teki çalışmada ajanlar toplu halde bir görseli oluşturmaya çalışırlar.…”

Section: Introductionunclassified

“…Bu çalışmayla benzer olarak 2-boyutlu bir ortam ve yapay sinir ağlarında CNN, FFNN katmanlar kullanmışlardır. [9]'daki çalışmada elde edilen sonuçlar incelendiğinde tek bir robotun navigasyon başarısı ~95%, çoklu robotların navigasyon başarısı ~90% olmuştur. Bu çalışmada 10.000 eğitim bölümü gerçekleştilmiştir ve DQN hiperparametreleri deneyim tekrarı hafızası 1.000.000, mini-batch sayısı ise 128 olarak belirlenmiştir.…”

Section: Introductionunclassified

See 1 more Smart Citation

Derin Takviyeli Öğrenme Tabanlı Bilinmeyen Ortamda Çoklu Robot Navigasyonu

Bar

¹

,

Karaköse

²

2022

Fırat Üniversitesi Mühendislik Bilimleri Dergisi

2

0

View full text Add to dashboard Cite

To solve multi-robot navigation with traditional methods, many algorithms can be used in combination, both for navigation and for the cooperation of these robots. These traditional methods using multiple algorithms are costly. Deep reinforcement learning (DRL) is simpler and less costly when compared to traditional methods. Nowadays, it is tried to solve real-world problems with DRL for these reasons. In this study, it has been tried to solve the multi-robot navigation problem with DRL. In the system in the proposed approach, there is a synchronous environment and more than one robot, target and obstacle in this environment. The robots in the environment move by selecting an action, respectively. At the same time, the robots as a dynamic obstacle for other robots. The robots try to reach their targets in the shortest path without any collision. At the same time, the robots try to plan paths so that they do not collide with another robot or extend the path of another robot. In order to provide these, multi-agent DQN algorithms, target-oriented state data, and reinforced adaptive reward mechanism were used. The system in the proposed approach was evaluated as the navigation success of a single robot, the navigation success of the multi-robot system, and the success rate according to the number of robots per unit square.

show abstract

Multi-Agent Pattern Formation: a Distributed Model-Free Deep Reinforcement Learning Approach

Cited by 9 publications

References 14 publications

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems

Derin Takviyeli Öğrenme Tabanlı Bilinmeyen Ortamda Çoklu Robot Navigasyonu

Contact Info

Product

Resources

About