A reinforcement learning-based predator-prey model

Wang, Xueting; Cheng, Jun; Wang, Lei

doi:10.1016/j.ecocom.2020.100815

Cited by 10 publications

(9 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Yet, it is important to distinguish the aim of these behavioural studies from the aim of applying RL to conservation management. In previous behavioural ecology studies, RL algorithms as a substitute for animal learning mechanisms (Perolat et al, 2017; Wang et al, 2020). When applying deep RL to conservation management, we do not make the assumption that an RL algorithm learns analogously to how an animal learns.…”

Section: Discussionmentioning

confidence: 99%

Deep reinforcement learning for conservation decisions

et al. 2022

View full text Add to dashboard Cite

1. Can machine learning help us make better decisions about a changing planet?In this paper, we illustrate and discuss the potential of a promising corner of machine learning known as deep reinforcement learning (RL) to help tackle the most challenging conservation decision problems. We provide a conceptual and technical introduction to deep RL as well as annotated code so that researchers can adopt, evaluate and extend these approaches.2. RL explicitly focuses on designing an agent who interacts with an environment that is dynamic and uncertain. Deep RL is the subfield of RL that incorporates deep neural networks into the agent. We train deep RL agents to solve sequential decision-making problems in setting fisheries quotas and managing ecological tipping points.3. We show that a deep RL agent is able to learn a nearly optimal solution for the fisheries management problem. For the tipping point problem, we show that a deep RL agent can outperform a sensible rule-of-thumb strategy. 4. Our results demonstrate that deep RL has the potential to solve challenging decision problems in conservation. While this potential may be compelling, the challenges involved in successfully deploying RL-based management to realistic scenarios are formidable-the required expertise and computational cost may place these applications beyond the reach of all but large, international technology firms. Ecologists must establish a better understanding of how these algorithms work and fail if we are to realize this potential and avoid the pitfalls such a transition would bring. We ultimately set forth a research framework based on well-posed, public challenges so that ecologists and computer scientists can collaborate towards solving hard decision-making problems in conservation.

show abstract

Section: Discussionmentioning

confidence: 99%

Deep reinforcement learning for conservation decisions

et al. 2022

View full text Add to dashboard Cite

show abstract

“…The problem of formalizing the co-evolution of predators and preys as a deep Q-learning-based reinforcement learning was recently introduced in Wang et al [ 36 ]. In addition, their subsequent work [ 37 ] adopted neural networks and presented a similar deep Q-learning-based evolution mechanism for predator-prey ecosystems, with discretized features for states. These deep Q-learning-based works rely on the update mechanism of Q in the following form:

…”

Section: Discussionmentioning

confidence: 99%

“…The problem of formalizing the co-evolution of predators and preys as RL was recently introduced in Wang et al [ 36 ], who claim that predators’ RL ability contributed to the stability of an ecosystem and helped predators attain more reasonable behavior patterns of coexistence with their prey; the RL effect of prey on its own population was not as successful as that of predators, and increased the risk of extinction of the predators. Their subsequent work [ 37 ] adopted neural networks and presented a similar RL-based evolution mechanism for predator-prey ecosystems, with discretized features for states. The methodologies used in these works are somewhat similar in essence to those of the present paper, but there are some distinctive differences in the final results, which are to be detailed in the discussion section below.…”

Section: Introductionmentioning

confidence: 99%

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

Park

Lee

Kim

et al. 2021

Entropy

View full text Add to dashboard Cite

The problem of finding adequate population models in ecology is important for understanding essential aspects of their dynamic nature. Since analyzing and accurately predicting the intelligent adaptation of multiple species is difficult due to their complex interactions, the study of population dynamics still remains a challenging task in computational biology. In this paper, we use a modern deep reinforcement learning (RL) approach to explore a new avenue for understanding predator-prey ecosystems. Recently, reinforcement learning methods have achieved impressive results in areas, such as games and robotics. RL agents generally focus on building strategies for taking actions in an environment in order to maximize their expected returns. Here we frame the co-evolution of predators and preys in an ecosystem as allowing agents to learn and evolve toward better ones in a manner appropriate for multi-agent reinforcement learning. Recent significant advancements in reinforcement learning allow for new perspectives on these types of ecological issues. Our simulation results show that throughout the scenarios with RL agents, predators can achieve a reasonable level of sustainability, along with their preys.

show abstract

“…On the other hand, reinforcement learning methods have driven impressive advances in artificial intelligence in recent years, surpassing human performance in many domains [22][23][24][25][26][27]. More recently, some researchers have begun to use reinforcement learning to model collective motion in a learning way [28][29][30][31]. An RL-based collective behavior model was proposed for the self-organized grouping of individuals, in which the reward function was constructed based on the distances between individuals [32].…”

Section: Related Workmentioning

confidence: 99%

Modeling Collective Behavior for Fish School With Deep Q-Networks

et al. 2023

View full text Add to dashboard Cite

Modeling collective behavior is a way to better understand the mechanisms that govern collective animal behaviors. Traditional rule-based modeling methods rely heavily on human prior knowledge and may not provide a proper explanation of the phenomenon of collective behaviors. This paper proposes a Deep Q-Networks (DQN)-based modeling method for fish school. Firstly, an individual's state (continuous value) is expressed by the angle between its direction and the average direction of its perceived neighbors. An individual's action is represented with discretized turning angle. Then, the reward function is constructed with the change in the number of neighbors. And finally, the neural network structure is constructed to represent the Q-value function and is trained by the DQN algorithm. The proposed approach is tested in two scenarios: single-learner and multi-learner. Results show that in both scenarios the proposed method can gradually converge and finally obtain a model that can produce collective behavior. On this basis, this paper also deeply analyzes the learned model from the perspectives of average order parameter and collective behavior patterns. It verifies that the behavior pattern generated by the learned model is a highly ordered collective behavior. In addition, we make a comparison between our proposed approach and the Q-Learning algorithm. The results show that our approach not only has a stronger ability to express policy and is better at handling continuous states but also has a more stable learning performance in training. INDEX TERMSCollective behavior, collective behavior model, Deep Q-Networks (DQN), fish school

show abstract

A reinforcement learning-based predator-prey model

Cited by 10 publications

References 37 publications

Deep reinforcement learning for conservation decisions

Deep reinforcement learning for conservation decisions

Co-Evolution of Predator-Prey Ecosystems by Reinforcement Learning Agents

Modeling Collective Behavior for Fish School With Deep Q-Networks

Contact Info

Product

Resources

About