Application of Deep Reinforcement Learning to UAV Fleet Control

Tožička, Jan; Szulyovszky, Benedek; Chambrier, Guillaume de; Sarwal, Varun; Wani, Umar; Gribulis, Mantas

doi:10.1007/978-3-030-01057-7_85

Cited by 8 publications

(8 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Examples of the scenarios include robot team navigation (Corke et al, 2005), smart grid operation (Dall'Anese et al, 2013), and control of mobile sensor networks (Cortes et al, 2004). Here we choose unmanned aerial vehicles (UAVs) (Yang and Liu, 2018;Pham et al, 2018;Tožička et al, 2018;Shamsoshoara et al, 2019;Cui et al, 2019;Qie et al, 2019), a recently surging application scenario of multi-agent autonomous systems, as one representative example. Specifically, a team of UAVs are deployed to accomplish a cooperation task, usually without the coordination of any central controller, i.e., in a decentralized fashion.…”

Section: Cooperative Setting Unmanned Aerial Vehiclesmentioning

confidence: 99%

“…Hence, the MADDPG algorithm proposed in Lowe et al ( 2017) is adopted, with centralized-learning-decentralized-execution. Two other tasks that can be tackled by MARL include resource allocation in UAV-enabled communication networks, using Q-learning based method (Cui et al, 2019), aerial surveillance and base defense in UAV fleet control, using policy optimization method in a purely centralized fashion (Tožička et al, 2018).…”

Section: Cooperative Setting Unmanned Aerial Vehiclesmentioning

confidence: 99%

See 1 more Smart Citation

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Zhang¹,

Yang²,

Başar³

2019

Preprint

154

View full text Add to dashboard Cite

Recent years have witnessed significant advances in reinforcement learning (RL), which has registered great success in solving various sequential decision-making problems in machine learning. Most of the successful RL applications, e.g., the games of Go and Poker, robotics, and autonomous driving, involve the participation of more than one single agent, which naturally fall into the realm of multi-agent RL (MARL), a domain with a relatively long history, and has recently re-emerged due to advances in single-agent RL techniques. Though empirically successful, theoretical foundations for MARL are relatively lacking in the literature. In this chapter, we provide a selective overview of MARL, with focus on algorithms backed by theoretical analysis. More specifically, we review the theoretical results of MARL algorithms mainly within two representative frameworks, Markov/stochastic games and extensive-form games, in accordance with the types of tasks they address, i.e., fully cooperative, fully competitive, and a mix of the two. We also introduce several significant but challenging applications of these algorithms. Orthogonal to the existing reviews on MARL, we highlight several new angles and taxonomies of MARL theory, including learning in extensive-form games, decentralized MARL with networked agents, MARL in the mean-field regime, (non-)convergence of policy-based methods for learning in games, etc. Some of the new angles extrapolate from our own research endeavors and interests. Our overall goal with this chapter is, beyond providing an assessment of the current state of the field on the mark, to identify fruitful future research directions on theoretical studies of MARL. We expect this chapter to serve as continuing stimulus for researchers interested in working on this exciting while challenging topic.Recent years have witnessed sensational advances of reinforcement learning (RL) in many prominent sequential decision-making problems, such as playing the game of Go (Silver

show abstract

Section: Cooperative Setting Unmanned Aerial Vehiclesmentioning

confidence: 99%

Section: Cooperative Setting Unmanned Aerial Vehiclesmentioning

confidence: 99%

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Zhang¹,

Yang²,

Başar³

2019

Preprint

154

View full text Add to dashboard Cite

show abstract

“…Furthermore, [108] formulates resource allocation in a downlink communication network as a SG and solved it using independent Q-learning. The work in [109] applies MARL for fleet control, particularly, aerial surveillance and base defense in a fully centralized fashion.…”

Section: B Marl For Uav-assisted Wireless Communicationsmentioning

confidence: 99%

Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial

Feriani¹,

Hossain²

2020

Preprint

View full text Add to dashboard Cite

Deep Reinforcement Learning (DRL) has recently witnessed significant advances that have led to multiple successes in solving sequential decision-making problems in various domains, particularly in wireless communications. The future sixth-generation (6G) networks are expected to provide scalable, low-latency, ultra-reliable services empowered by the application of data-driven Artificial Intelligence (AI). The key enabling technologies of future 6G networks, such as intelligent metasurfaces, aerial networks, and AI at the edge, involve more than one agent which motivates the importance of multi-agent learning techniques. Furthermore, cooperation is central to establishing self-organizing, self-sustaining, and decentralized networks. In this context, this tutorial focuses on the role of DRL with an emphasis on deep Multi-Agent Reinforcement Learning (MARL) for AI-enabled 6G networks. The first part of this paper will present a clear overview of the mathematical frameworks for single-agent RL and MARL. The main idea of this work is to motivate the application of RL beyond the model-free perspective which was extensively adopted in recent years. Thus, we provide a selective description of RL algorithms such as Model-Based RL (MBRL) and cooperative MARL and we highlight their potential applications in 6G wireless networks. Finally, we overview the state-of-the-art of MARL in fields such as Mobile Edge Computing (MEC), Unmanned Aerial Vehicles (UAV) networks, and cell-free massive MIMO, and identify promising future research directions. We expect this tutorial to stimulate more research endeavors to build scalable and decentralized systems based on MARL.

show abstract

“…Next, take task point 17 as an example. Before the inspection of task point 17, UAV1, UAV2, and UAV3 were assigned patrol sequences (2,5,6,8,1,12,13,14), (11,15,10,20), and (18, 4, 7, 9, 19, 3) respectively. After inspecting task point 14, the total time consumption of UAV1 reached 55.006.…”

Section: : If (T Kmentioning

confidence: 99%

Task Assignment Algorithm for Road Patrol by Multiple UAVs With Multiple Bases and Rechargeable Endurance

et al. 2019

View full text Add to dashboard Cite

Increasing attention has been paid to the application of Unmanned Aerial Vehicles (UAVs) in traffic field. Existing studies on UAVs in road patrol task assignment rarely integrate the actual requirements of multiple bases, charging problems during inspection and multiple types of UAVs into the patrol model, and rarely consider the fastest time to complete the patrol of all task points as the optimization target. In addition, most existing solving algorithms are difficult to use to directly solve problem models meeting the above requirements and objectives, and some existing algorithms need to be improved in terms of solving quality. In order to solve those problems, a task assignment model for road patrol by multiple UAVs with multiple bases and rechargeable endurance is established, and a time-priority immune clonal selection algorithm with optimization-revision is proposed. The optimal sequence of task points is obtained by the immune clonal selection algorithm, and the time-priority method is adopted to divide the sequence of task points. The optimal UAV paths are further optimized and modified. With 20 task points, the experimental results show that the earliest completion time obtained by the proposed algorithm decreased by 1.071 and 2.1209 compared with the earliest reachable time particle swarm optimization algorithm and the improved dynamic programming ant colony algorithm, respectively. The results indicate that proposed algorithm achieves better performance in terms of solution quality. In addition, experiments with 50 and 100 task points show that the algorithm is also suitable for medium and large scale problems. INDEX TERMSImmune clonal selection algorithm, optimization-revision, rechargeable endurance, road patrol, time priority, UAV. I. INTRODUCTION Unmanned aerial vehicles (UAVs) have made remarkable achievements in the military [1], [2] and have been widely used in various civilian fields as well [3]-[5]. In recent years, UAVs have shown a broad application prospects in the field of transportation. In the future, it will play an important role in urban traffic planning, road patrol, bridge inspection, traffic dredging, accident emergency treatment, traffic accident forensics, criminal tracking and other transportation aspects [6]-[8]. Due to its flexible control, free from traffic restrictions on the road, wide patrol range and lack of a need for manual intervention in the whole process. UAVs can The associate editor coordinating the review of this manuscript and approving it for publication was Ying Li. LINHUI CHENG (M'19) was born in Xiangyang, Hubei, China, in 1980. She received the B.S. degree in computer software from Central China Normal University, Wuhan, China, in 2002, and the M.S. degree in computer science and technology from South-Central University for Nationalities, Wuhan, in 2008. She is currently pursuing the Ph.D. degree in computer science and technology with the

show abstract

Application of Deep Reinforcement Learning to UAV Fleet Control

Cited by 8 publications

References 1 publication

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms

Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial

Task Assignment Algorithm for Road Patrol by Multiple UAVs With Multiple Bases and Rechargeable Endurance

Contact Info

Product

Resources

About