Learning to Fly: A Distributed Deep Reinforcement Learning Framework for Software-Defined UAV Network Control

Abstract

Control and performance optimization of wireless networks of Unmanned Aerial Vehicles (UAVs) require scalable approaches that go beyond architectures based on centralized network controllers. At the same time, the performance of model-based optimization approaches is often limited by the accuracy of the approximations and relaxations necessary to solve the UAV network control problem through convex optimization or similar techniques, and by the accuracy of the channel network models used. To address these challenges, this article introduces a new architectural framework to control and optimize UAV networks based on Deep Reinforcement Learning (DRL). Furthermore, it proposes a virtualized, 'ready-to-fly' emulation environment to generate the extensive wireless data traces necessary to train DRL algorithms, which are notoriously hard to generate and collect on battery-powered UAV networks.The training environment integrates previously developed wireless protocol stacks for UAVs into the CORE/EMANE emulation tool. Our 'ready-to-fly' virtual environment guarantees scalable collection of high-fidelity wireless traces that can be used to train DRL agents. The proposed DRL architecture enables distributed data-driven optimization (with up to 3.7x throughput improvement and 0.2x latency reduction in reported experiments), facilitates network reconfiguration, and provides a scalable solution for large UAV networks.

Drone Tracking with Drone using Deep Learning

“…The parameter sharing method is suggested in the training for swarm robots by testing the PPO algorithm in different environments [11]. A new distributed reinforcement learning architecture is presented in many other UAV studies [12]. In this study, the control and optimization of UAVs are aimed.…”

Software-Defined UAV Networks for 6G Systems: Requirements, Opportunities, Emerging Techniques, Challenges, and Research Directions

“…Additionally, the SDUAV networks can provide dynamic network orchestration, seamless mobility, and effective resource management. Through optimized intelligent decision-making ability and resource allocation, the SDUAV networks can adapt in real-time to meet the specific requirements of different applications, resulting in enhanced overall network efficiency and performance [31]. At the same time, centralized controllability of the SDUAV networks can simplify the integration of heterogeneous networks, ensures seamless connectivity and enhances interaction routes from end to end [32].…”