A learning-based flexible autonomous motion control method for UAV in dynamic unknown environments

Abstract

This paper presents a deep reinforcement learning (DRL)-based motion control method to provide unmanned aerial vehicles (UAVs) with additional flexibility while flying across dynamic unknown environments autonomously. This method is applicable in both military and civilian fields such as penetration and rescue. The autonomous motion control problem is addressed through motion planning, action interpretation, trajectory tracking, and vehicle movement within the DRL framework. Novel DRL algorithms are presented by combining two difference-amplifying approaches with traditional DRL methods and are used for solving the motion planning problem. An improved Lyapunov guidance vector field (LGVF) method is used to handle the trajectory-tracking problem and provide guidance control commands for the UAV. In contrast to conventional motion-control approaches, the proposed methods directly map the sensorbased detections and measurements into control signals for the inner loop of the UAV, i.e., an end-to-end control. The training experiment results show that the novel DRL algorithms provide more than a 20% performance improvement over the state-ofthe-art DRL algorithms. The testing experiment results demonstrate that the controller based on the novel DRL and LGVF, which is only trained once in a static environment, enables the UAV to fly autonomously in various dynamic unknown environments. Thus, the proposed technique provides strong flexibility for the controller.

A Reinforcement Learning Method Based on an Improved Sampling Mechanism for Unmanned Aerial Vehicle Penetration

“…(2) Based on the reinforcement learning network. For example, Yuxie Luo [12] established an intelligent UAV model using the reinforcement learning network and built a reward function using the cooperative parameters of multiple UAVs to guide UAVs to conduct collaborative penetration; Yue Li [13] used reinforcement learning algorithms for training in four scenarios, frontal attack, escape, pursuit, and energy storage, thereby improving the intelligent decision-making level of air confrontation; Kaifang Wan [14] proposed a motion control method based on deep reinforcement learning (DRL), which provides additional flexibility for UAV penetration within the DRL framework; Liang Li [15] mainly focused on the winning region of three players in the reconnaissance penetration game, proposed an explicit policy method for analyzing and constructing barriers, and provided a complete solution by integrating the games of kind and degree.…”

UAV Maneuvering Decision-Making Algorithm Based on Deep Reinforcement Learning Under the Guidance of Expert Experience

“…For instance, some people used UAVs to carry out delivery of relief supplies, extinguishing, and so on [3][4][5]. Consequently, it has become one of the key issues for engineering applications to improve the autonomous flight capability of UAV [6][7][8][9].…”

Implementation of a Long Short-Term Memory Neural Network-Based Algorithm for Dynamic Obstacle Avoidance

“…The model was evaluated through a series of simulations, which showed that the developed strategy was efficient in avoiding collisions with obstacles in complex dynamic environments and narrow spaces. Wan et al [11] presented a deep reinforcement learning (DRL)-based method for motion control of unmanned aerial vehicles (UAVs) while flying autonomously through unknown dynamic environments providing good adaptability and navigation through them. Wang et al [12] proposed a globally guided reinforcement learning approach (G2RL) that incorporated a novel reward structure that generalized to arbitrary environments and that, when applied to solve the multi-robot path-planning problem, proved to be robust, scalable, and generalizable and outperformed existing distributed multi-robot path-planning methods.…”