Emergent Escape-based Flocking behavior using Multi-Agent Reinforcement Learning

Hahn, Carsten; Phan, Thomy; Gabor, Thomas; Belzner, Lenz; Linnhoff‐Popien, Claudia

doi:10.1162/isal_a_00226

Cited by 8 publications

(13 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Further, Sunehag et al (2019) find the emergence of flocking and symbiosis with rewards shaping by independent MARL in simulated multiple-species ecosystems. In parallel, Hahn et al (2019) demonstrate the emergence of swarming without reward shaping and later show that in their scenario, prey swarming is a Nash Equilibrium (Hahn et al, 2020) and the prey could perform better if collectively fleeing independently. Ritz et al (2020) also found independently fleeing prey to be significantly harder to hunt for a single RL predator.…”

Section: Further Emergent Behaviormentioning

confidence: 91%

See 1 more Smart Citation

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

Ritz¹,

Ratke²,

Phan³

et al. 2021

The 2021 Conference on Artificial Life

Self Cite

View full text Add to dashboard Cite

This paper considers sustainable and cooperative behavior in multi-agent systems. In the proposed predator-prey simulation, multiple selfish predators can learn to act sustainably by maintaining a herd of reproducing prey and further hunt cooperatively for long term benefit. Since the predators face starvation pressure, the scenario can also turn in a tragedy of the commons if selfish individuals decide to greedily hunt down the prey population before their conspecifics do, ultimately leading to extinction of prey and predators. This paper uses Multi-Agent Reinforcement Learning to overcome a collapse of the simulated ecosystem, analyzes the impact factors over multiple dimensions and proposes suitable metrics. We show that up to three predators are able to learn sustainable behavior in form of collective herding under starvation pressure. Complex cooperation in form of group hunting emerges between the predators as their speed is handicapped and the prey is given more degrees of freedom to escape. The implementation of environment and reinforcement learning pipeline is available online. 1

show abstract

Section: Further Emergent Behaviormentioning

confidence: 91%

“…Regarding the agents, the prey strategies proposed by Hahn et al (2019) were compared in preliminary experiments (c.f. Figure 2a).…”

Section: Methodsmentioning

confidence: 99%

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

Ritz¹,

Ratke²,

Phan³

et al. 2021

The 2021 Conference on Artificial Life

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multi-agent reinforcement learning (MARL) has become popular to model individually rational agents in SDs and SSDs to examine emergent behavior [6,23,31,15,36]. The goal of each agent is defined by an individual reward function.…”

Section: Introductionmentioning

confidence: 99%

Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning

Phan

Sommer

Ritz

et al. 2022

Preprint

View full text Add to dashboard Cite

Peer incentivization (PI) is a recent approach, where all agents learn to reward or to penalize each other in a distributed fashion which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly integrated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information which limits scalability and applicability to real-world scenarios, where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to mutually exchange acknowledgment tokens to shape individual rewards. Each agent evaluates the monotonic improvement of its individual situation in order to accept or reject acknowledgment requests from other agents. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can defect from the protocol and where communication failures can occur. We also evaluate the sensitivity of MATE w.r.t. the choice of token values.

show abstract

“…After the decentralized planning procedure is over, the trained policy is reintegrated into the method. Hahn et al [ 32 ] proposed Swarm Emergent Learning Fish (SELFish), a reinforcement learning approach to evolve prey animals based on predation. For the multi-agent concept, every individual agent optimizes its own behavior without any centralization or integration.…”

Section: Introductionmentioning

confidence: 99%

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

Park

Lee

Kim

et al. 2021

Entropy

View full text Add to dashboard Cite

The problem of finding adequate population models in ecology is important for understanding essential aspects of their dynamic nature. Since analyzing and accurately predicting the intelligent adaptation of multiple species is difficult due to their complex interactions, the study of population dynamics still remains a challenging task in computational biology. In this paper, we use a modern deep reinforcement learning (RL) approach to explore a new avenue for understanding predator-prey ecosystems. Recently, reinforcement learning methods have achieved impressive results in areas, such as games and robotics. RL agents generally focus on building strategies for taking actions in an environment in order to maximize their expected returns. Here we frame the co-evolution of predators and preys in an ecosystem as allowing agents to learn and evolve toward better ones in a manner appropriate for multi-agent reinforcement learning. Recent significant advancements in reinforcement learning allow for new perspectives on these types of ecological issues. Our simulation results show that throughout the scenarios with RL agents, predators can achieve a reasonable level of sustainability, along with their preys.

show abstract

Emergent Escape-based Flocking behavior using Multi-Agent Reinforcement Learning

Cited by 8 publications

References 8 publications

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

A Sustainable Ecosystem through Emergent Cooperation in Multi-Agent Reinforcement Learning

Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

Contact Info

Product

Resources

About