Collaborative Coverage Path Planning of UAV Cluster based on Deep Reinforcement Learning

Dong, Zanliang; Liu, Chunhui

doi:10.1109/icftic54370.2021.9647298

Cited by 13 publications

(7 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The author in [4] optimize the data transmission mode and the trajectory of two UAVs to improve the performance of the communication system. Additionally, [5] investigate the optimal path to travel all points in a target area under time and energy constraints. Different from the above, this paper investigates an obstacles avoidance route from the start point to the destination for each UAV to ensure the completion of the distribution task as a premise.…”

Section: A Problem Statementmentioning

confidence: 99%

A Two-Stage Approach of Joint Route Planning and Resource Allocation for Multiple UAVs in Unmanned Logistics Distribution

2022

Self Cite

View full text Add to dashboard Cite

Unmanned aerial vehicles (UAVs) can serve as means of delivery to enhance the effectiveness and accuracy of logistics distribution in various scenarios, such as emergency items delivery for disaster areas and logistics services for remote areas. This paper focuses on the problem of route planning for UAVs, which is the basis for UAVs to complete distribution tasks. Besides, to enhance the collaboration of UAVs during the delivery mission, this work also studies wireless resource (spectrum, power) allocation problem for multiple UAVs enabled communication networks. Due to the non-convex and combinatorial characteristics, it is challenging to obtain an optimal strategy for joint route planning and resource allocation issues within a finite time. To this end, the paths of UAVs are planned by the simulated annealing (SA) method in the first stage. Based on the preplanned paths, a double deep Q network (DDQN) based resource allocation method is proposed to maximize the average sum capacity of UAV-to-UAV (U2U) links and reduce transmission delay while minimizing the interference to UAV-to-infrastructure (U2I) links. Multi-UAV communication networks are reconnected according to the positions of UAVs changing in each time slot. Then each U2U link acting as an agent learns to improve spectrum and power allocation policy with imperfect knowledge of the environment. Simulation results demonstrate that the proposed DDQN-based resource management scheme can achieve higher system communication capacity than a random scheme. Moreover, the successful transmission probability of U2U links obtained by the DDQN-based method is much higher than a Particle Swarm Optimization (PSO) based method.

show abstract

Section: A Problem Statementmentioning

confidence: 99%

A Two-Stage Approach of Joint Route Planning and Resource Allocation for Multiple UAVs in Unmanned Logistics Distribution

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…DRL-based approaches can tackle CPP problems directly and without the need for previous decomposition but are generally underexplored in the context of autonomous UAV coverage missions. One of the rare examples is given in [13], where a Q-learning-based algorithm is proposed for coordinating multiple UAVs on a coverage mission. In contrast to the scenario we investigate, the coverage mission is set in an environment not requiring obstacle avoidance or recharge.…”

Section: Related Workmentioning

confidence: 99%

“…CPP is a fundamental problem with broad applications, holding the potential to address numerous existing challenges across various domains. However, only a few papers investigated DRL-based methods specifically tailored to the complexities of the UAV coverage problem [11]- [13], or investigated the general applicability of DRL for ground-based CPP [14]- [20]. In this work, we present several contributions to address these gaps:…”

Section: Introductionmentioning

confidence: 99%

UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learning

Theile

Bayerlein

Nai

et al. 2020

2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

View full text Add to dashboard Cite

Coverage path planning (CPP) is a critical problem in robotics, where the goal is to find an efficient path that covers every point in an area of interest. This work addresses the power-constrained CPP problem with recharge for batterylimited unmanned aerial vehicles (UAVs). In this problem, a notable challenge emerges from integrating recharge journeys into the overall coverage strategy, highlighting the intricate task of making strategic, long-term decisions. We propose a novel proximal policy optimization (PPO)-based deep reinforcement learning (DRL) approach with map-based observations, utilizing action masking and discount factor scheduling to optimize coverage trajectories over the entire mission horizon. We further provide the agent with a position history to handle emergent state loops caused by the recharge capability. Our approach outperforms a baseline heuristic, generalizes to different target zones and maps, with limited generalization to unseen maps. We offer valuable insights into DRL algorithm design for longhorizon problems and provide a publicly available software framework for the CPP problem.

show abstract

“…A full-coverage path planning algorithm based on Q-Learning is proposed in [23], which optimizes coverage paths using raster graphs. Robot reward and punishment mechanism in DQN (Deep Q-Network) is proposed in Ref [24] for fullcoverage of UAV. Jin et al [25] use reinforcement learning to achieve coverage of three-dimensional object representation, and demonstrate that ϵ-greedy strategy is better than pure greedy strategy.…”

Section: Introductionmentioning

confidence: 99%

Predator-Prey Reward Based Q-Learning Coverage Path Planning for Mobile Robot

2023

View full text Add to dashboard Cite

Coverage Path Planning (CPP in short) is a basic problem for mobile robot when facing a variety of applications. Q-Learning based coverage path planning algorithms are beginning to be explored recently. To overcome the problem of traditional Q-Learning of easily falling into local optimum, in this paper, the new-type reward functions originating from Predator-Prey model are introduced into traditional Q-Learning based CPP solution, which introduces a comprehensive reward function that incorporates three rewards including Predation Avoidance Reward Function, Smoothness Reward Function and Boundary Reward Function. In addition, the influence of weighting parameters on the total reward function is discussed. Extensive simulation results and practical experiments verify that the proposed Predator-Prey reward based Q-Learning Coverage Path Planning (PP-Q-Learning based CPP in short) has better performance than traditional BCD and Q-Learning based CPP in terms of repetition ratio and turns number.

show abstract

Collaborative Coverage Path Planning of UAV Cluster based on Deep Reinforcement Learning

Cited by 13 publications

References 9 publications

A Two-Stage Approach of Joint Route Planning and Resource Allocation for Multiple UAVs in Unmanned Logistics Distribution

A Two-Stage Approach of Joint Route Planning and Resource Allocation for Multiple UAVs in Unmanned Logistics Distribution

UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learning

Predator-Prey Reward Based Q-Learning Coverage Path Planning for Mobile Robot

Contact Info

Product

Resources

About