Q learning algorithm based UAV path learning and obstacle avoidence approach

Zhao, Yijing; Zhang, Zheng; Zhang, Xiaoyi; Yang, Liu

doi:10.23919/chicc.2017.8027884

Cited by 82 publications

(35 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome the shortcomings of the methods described above, other studies [16][17][18][19] have proposed using reinforcement learning (RL), an online learning method, for UAV motion planning. Reinforcement learning can solve the UAV's AMP problem with navigating in an unknown environment, but as the environment becomes more complex and closer to reality, the "dimension curse" problem limits its development.…”

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV’s Autonomous Motion Planning in Complex Unknown Environments

Wan

Gao

et al. 2020

Sensors

View full text Add to dashboard Cite

Autonomous motion planning (AMP) of unmanned aerial vehicles (UAVs) is aimed at enabling a UAV to safely fly to the target without human intervention. Recently, several emerging deep reinforcement learning (DRL) methods have been employed to address the AMP problem in some simplified environments, and these methods have yielded good results. This paper proposes a multiple experience pools (MEPs) framework leveraging human expert experiences for DRL to speed up the learning process. Based on the deep deterministic policy gradient (DDPG) algorithm, a MEP–DDPG algorithm was designed using model predictive control and simulated annealing to generate expert experiences. On applying this algorithm to a complex unknown simulation environment constructed based on the parameters of the real UAV, the training experiment results showed that the novel DRL algorithm resulted in a performance improvement exceeding 20% as compared with the state-of-the-art DDPG. The results of the experimental testing indicate that UAVs trained using MEP–DDPG can stably complete a variety of tasks in complex, unknown environments.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV’s Autonomous Motion Planning in Complex Unknown Environments

Wan

Gao

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…To evaluate the performance of the proposed QL-GD, the comparison with CQL in mobile robot path planning was conducted. The experiment was carried out in simulated environment with various obstacle patterns in four different maps in MATLAB software by referring [29]. Size of all maps were 20 x 20 unit grid.…”

Section: Experiments Setupmentioning

confidence: 99%

Mobile Robot Path Planning using Q-Learning with Guided Distance

Low

Ong

Low

2018

IJET

View full text Add to dashboard Cite

In path planning for mobile robot, classical Q-learning algorithm requires high iteration counts and longer time taken to achieve convergence. This is due to the beginning stage of classical Q-learning for path planning consists of mostly exploration, involving random direction decision making. This paper proposed the addition of distance aspect into direction decision making in Q-learning. This feature is used to reduce the time taken for the Q-learning to fully converge. In the meanwhile, random direction decision making is added and activated when mobile robot gets trapped in local optima. This strategy enables the mobile robot to escape from local optimal trap. The results show that the time taken for the improved Q-learning with distance guiding to converge is longer than the classical Q-learning. However, the total number of steps used is lower than the classical Q-learning.

show abstract

“…In this method, it combines greedy strategy with Boltzmann strategy to improve exploration. Yijing et al [13] proposes an adaptive random exploration Q-learning method which designs the learning, escape and action modules of UAV respectively. Yan et al [14] initializes Q matrix with environment and target, and adds the function of avoiding repetitive actions in the initial stage of training.…”

Section: Introductionmentioning

confidence: 99%

Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning

2020

IEEE Access

120

View full text Add to dashboard Cite

In this paper, we focus on the study of UAV ground target tracking under obstacle environments using deep reinforcement learning, and an improved deep deterministic policy gradient(DDPG) algorithm is presented. A reward function based on line of sight and artificial potential field is constructed to guide the behavior of UAV to achieve target tracking, and a penalty term of action makes the trajectory smooth. In order to improve the exploration ability, multiple UAVs, which controlled by the same policy network, are used to perform tasks in each episode. Taking into account that the history observations have a great degree of correlation with the policy, long short-term memory networks are used to approximate the state of environments, which improve the approximation accuracy and the efficiency of data utilization. The simulation results show that the propose method can make the UAV keep target tracking and obstacle avoidance effectively.INDEX TERMS DDPG, deep reinforcement learning, obstacle avoidance, target tracking, UAV.

show abstract

Q learning algorithm based UAV path learning and obstacle avoidence approach

Cited by 82 publications

References 16 publications

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV’s Autonomous Motion Planning in Complex Unknown Environments

Deep Reinforcement Learning Approach with Multiple Experience Pools for UAV’s Autonomous Motion Planning in Complex Unknown Environments

Mobile Robot Path Planning using Q-Learning with Guided Distance

Path Planning for UAV Ground Target Tracking via Deep Reinforcement Learning

Contact Info

Product

Resources

About