Maneuvering target tracking of UAV based on MN-DDPG and transfer learning

Li, Bo; Yang, Zhen; Chen, Daqing; Liang, Shiyang; Ma, Hao

doi:10.1016/j.dt.2020.11.014

Cited by 92 publications

(28 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the work of [41], a DDPG-based control framework was used to provide learning and autonomous decision-making capability for UAVs. In addition, an improved method, named mixture noise DDPG (MN-DDPG), for introducing a type of mixed noises to assist UAV by exploring stochastic strategies for optimal online planning was proposed.…”

Section: Literature Surveymentioning

confidence: 99%

Twin Delayed Deep Deterministic Policy Gradient-Based Target Tracking for Unmanned Aerial Vehicle With Achievement Rewarding and Multistage Training

et al. 2022

View full text Add to dashboard Cite

show abstract

Section: Literature Surveymentioning

confidence: 99%

Twin Delayed Deep Deterministic Policy Gradient-Based Target Tracking for Unmanned Aerial Vehicle With Achievement Rewarding and Multistage Training

et al. 2022

View full text Add to dashboard Cite

show abstract

“…As shown in Figure 2, when formulating real-time tracks, owing to the constraints of aircraft mobility and dual-aircraft cooperation, the nearest distance to the radar detection range min Although many inertial navigations equipped with auxiliary autopilots can provide high-precision dynamic information [24], ultrasonic rangefinders cannot meet the…”

Section: Map Descriptionmentioning

confidence: 99%

“…As shown in Figure 2 Although many inertial navigations equipped with auxiliary autopilots can provide high-precision dynamic information [24], ultrasonic rangefinders cannot meet the Although many inertial navigations equipped with auxiliary autopilots can provide high-precision dynamic information [24], ultrasonic rangefinders cannot meet the require-ments of detection distance in the combat environment. Therefore, phased-array radar is selected as the distance sensor.…”

Section: Map Descriptionmentioning

confidence: 99%

Deep Reinforcement Learning for Intelligent Dual-UAV Reconnaissance Mission Planning

et al. 2022

View full text Add to dashboard Cite

The reconnaissance of high-value targets is prerequisite for effective operations. The recent appreciation of deep reinforcement learning (DRL) arises from its success in navigation problems, but due to the competitiveness and complexity of the military field, the applications of DRL in the military field are still unsatisfactory. In this paper, an end-to-end DRL-based intelligent reconnaissance mission planning is proposed for dual unmanned aerial vehicle (dual UAV) cooperative reconnaissance missions under high-threat and dense situations. Comprehensive consideration is given to specific mission properties and parameter requirements through the whole modelling. Firstly, the reconnaissance mission is described as a Markov decision process (MDP), and the mission planning model based on DRL is established. Secondly, the environment and UAV motion parameters are standardized to input the neural network, aiming to deduce the difficulty of algorithm convergence. According to the concrete requirements of non-reconnaissance by radars, dual-UAV cooperation and wandering reconnaissance in the mission, four reward functions with weights are designed to enhance agent understanding to the mission. To avoid sparse reward, the clip function is used to control the reward value range. Finally, considering the continuous action space of reconnaissance mission planning, the widely applicable proximal policy optimization (PPO) algorithm is used in this paper. The simulation is carried out by combining offline training and online planning. By changing the location and number of ground detection areas, from 1 to 4, the model with PPO can maintain 20% of reconnaissance proportion and a 90% mission complete rate and help the reconnaissance UAV to complete efficient path planning. It can adapt to unknown continuous high-dimensional environmental changes, is generalizable, and reflects strong intelligent planning performance.

show abstract

“…The observation model of the agent was presented to provide the agent with the ability to sense the environment [30]. In this multi-agent pursuit-evasion task, (x i a , y i a ) is the position of each agent in the pursuit formation, and both (x j e , y j e ) and (x k o , y k o ) represent the position of the center point of the evader and the threatened area, respectively.…”

Section: Observation Modelmentioning

confidence: 99%

An Improved Approach towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Wan

Zhai

et al. 2021

Entropy

Self Cite

View full text Add to dashboard Cite

A pursuit–evasion game is a classical maneuver confrontation problem in the multi-agent systems (MASs) domain. An online decision technique based on deep reinforcement learning (DRL) was developed in this paper to address the problem of environment sensing and decision-making in pursuit–evasion games. A control-oriented framework developed from the DRL-based multi-agent deep deterministic policy gradient (MADDPG) algorithm was built to implement multi-agent cooperative decision-making to overcome the limitation of the tedious state variables required for the traditionally complicated modeling process. To address the effects of errors between a model and a real scenario, this paper introduces adversarial disturbances. It also proposes a novel adversarial attack trick and adversarial learning MADDPG (A2-MADDPG) algorithm. By introducing an adversarial attack trick for the agents themselves, uncertainties of the real world are modeled, thereby optimizing robust training. During the training process, adversarial learning was incorporated into our algorithm to preprocess the actions of multiple agents, which enabled them to properly respond to uncertain dynamic changes in MASs. Experimental results verified that the proposed approach provides superior performance and effectiveness for pursuers and evaders, and both can learn the corresponding confrontational strategy during training.

show abstract

Maneuvering target tracking of UAV based on MN-DDPG and transfer learning

Cited by 92 publications

References 21 publications

Twin Delayed Deep Deterministic Policy Gradient-Based Target Tracking for Unmanned Aerial Vehicle With Achievement Rewarding and Multistage Training

Twin Delayed Deep Deterministic Policy Gradient-Based Target Tracking for Unmanned Aerial Vehicle With Achievement Rewarding and Multistage Training

Deep Reinforcement Learning for Intelligent Dual-UAV Reconnaissance Mission Planning

An Improved Approach towards Multi-Agent Pursuit–Evasion Game Decision-Making Using Deep Reinforcement Learning

Contact Info

Product

Resources

About