Neural Q Learning Algorithm based UAV Obstacle Avoidance

Zhou, Benchun; Wang, Weihong; Wang, Zhifeng; Ding, Baoyang

doi:10.1109/gncc42960.2018.9018854

Cited by 6 publications

(7 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Navigation may be challenging for the UAV during missions since it may not be aware of the precise environment and obstacle information. Therefore, Benchun Zhou et al helped the UAV avoid obstacles by studying the neural Q-learning method [2]. They assumed that the environment in which the UAV was located was unknown, and when there were obstacles within the execution range, In order to address the quality issue with the Q-table, maintain the route planning process, and take into account the UAV's physical limitations, they integrated BP neural networks to Q-learning.…”

Section: Uav Obstacle Avoidance Based On Q-learningmentioning

confidence: 99%

See 1 more Smart Citation

Q-Learning: Applications and Convergence Rate Optimization

Liu

2023

HSET

View full text Add to dashboard Cite

As an important algorithm of artificial intelligence technology, Q-learning algorithm plays a significant part in a number of fields, such as driverless technology, industrial automation, health care, intelligent search, game, etc. As a classical learning algorithm in reinforcement learning, with the help of an experienced action sequence in a Markov environment, an agent can learn to select the best course of action using the model-free learning technique known as Q-learning. This paper mainly discusses the addition of received signal strength (RSS) to the Q-learning algorithm to navigate unmanned aerial vehicle (UAV), summarizes the main content and results of the neural Q-learning algorithm helping UAV avoid obstacles, adaptive and random exploration (ARE) method is proposed to address the issue in UAV planning a route tasks, summarizes the content and results of route designing of moving robot using obstacle characteristics as Q-learning states and actions, the Q-learning algorithm employs a novel exploration technique that combines ε-rapacious exploring with Boltzmann theory. to help mobile robot to plan its path, and analyzes the convergence speed of the algorithm for planning a route of stage Q-learning and the path planning algorithm of traditional Q-learning. When there are many states and actions, the operation efficiency of the Q-learning algorithm will be greatly reduced, so it is necessary to study in depth how to reduce the operation time of the algorithm for Q-learning and increase the convergence velocity of algorithm for Q-learning.

show abstract

Section: Uav Obstacle Avoidance Based On Q-learningmentioning

confidence: 99%

“…𝑄(𝑠, 𝑎) = 𝑅(𝑠, 𝑎) + 𝛾 ∑ 𝑝(𝑠, 𝑎, 𝑠 ′ ) ∑ 𝑄(𝑠 ′ , 𝑎 ′ ) 𝑎 ′ ∈𝐴 𝑠 ′ ∈𝑆 (2) Highlights in Science, Engineering and Technology…”

Section: Mobile Robot Path Planning Based On Q-learningmentioning

confidence: 99%

Q-Learning: Applications and Convergence Rate Optimization

Liu

2023

HSET

View full text Add to dashboard Cite

show abstract

“…In recent years, reinforcement learning (RL) based approaches have been widely investigated in the UAV navigation domain [46][47][48][49][50][51][52][53][54]. The classic Q-learning (CQL) algorithm proposed in [55] has the underlying principle that when the UAV observes the environment information at time step k and takes actions based on the environment information obtained, an immediate reward can be obtained from the environment.…”

Section: Related Workmentioning

confidence: 99%

“…A neural Q learning (NQL) based approach is proposed in [51]. The authors combine the CQL with the Back Propagation Neural Network (BPN) to obtain the resulting NQL, which can be trained to achieve the obstacle avoidance purpose.…”

Section: Related Workmentioning

confidence: 99%

A 3D Vision Cone Based Method for Collision Free Navigation of a Quadcopter UAV among Moving Obstacles

Ming

Huang

2021

Drones

View full text Add to dashboard Cite

In the near future, it’s expected that unmanned aerial vehicles (UAVs) will become ubiquitous surrogates for human-crewed vehicles in the field of border patrol, package delivery, etc. Therefore, many three-dimensional (3D) navigation algorithms based on different techniques, e.g., model predictive control (MPC)-based, navigation potential field-based, sliding mode control-based, and reinforcement learning-based, have been extensively studied in recent years to help achieve collision-free navigation. The vast majority of the 3D navigation algorithms perform well when obstacles are sparsely spaced, but fail when facing crowd-spaced obstacles, which causes a potential threat to UAV operations. In this paper, a 3D vision cone-based reactive navigation algorithm is proposed to enable small quadcopter UAVs to seek a path through crowd-spaced 3D obstacles to the destination without collisions. The proposed algorithm is simulated in MATLAB with different 3D obstacles settings to demonstrate its feasibility and compared with the other two existing 3D navigation algorithms to exhibit its superiority. Furthermore, a modified version of the proposed algorithm is also introduced and compared with the initially proposed algorithm to lay the foundation for future work.

show abstract

“…Ref. [15] using Q-learning employs adaptive random exploration to achieve UAV navigation and obstacle avoidance, but the method is constrained by action space and sample dimensions. Addressing the issue of UAVs' inability to acquire environmental information, ref.…”

Section: Introductionmentioning

confidence: 99%

Dynamic Scene Path Planning of UAVs Based on Deep Reinforcement Learning

Tang,

Liang,

2024

Drones

View full text Add to dashboard Cite

Traditional unmanned aerial vehicle path planning methods focus on addressing planning issues in static scenes, struggle to balance optimality and real-time performance, and are prone to local optima. In this paper, we propose an improved deep reinforcement learning approach for UAV path planning in dynamic scenarios. Firstly, we establish a task scenario including an obstacle assessment model and model the UAV’s path planning problem using the Markov Decision Process. We translate the MDP model into the framework of reinforcement learning and design the state space, action space, and reward function while incorporating heuristic rules into the action exploration policy. Secondly, we utilize the Q function approximation of an enhanced D3QN with a prioritized experience replay mechanism and design the algorithm’s network structure based on the TensorFlow framework. Through extensive training, we obtain reinforcement learning path planning policies for both static and dynamic scenes and innovatively employ a visualized action field to analyze their planning effectiveness. Simulations demonstrate that the proposed algorithm can accomplish UAV dynamic scene path planning tasks and outperforms classical methods such as A*, RRT, and DQN in terms of planning effectiveness.

show abstract

Neural Q Learning Algorithm based UAV Obstacle Avoidance

Cited by 6 publications

References 8 publications

Q-Learning: Applications and Convergence Rate Optimization

Q-Learning: Applications and Convergence Rate Optimization

A 3D Vision Cone Based Method for Collision Free Navigation of a Quadcopter UAV among Moving Obstacles

Dynamic Scene Path Planning of UAVs Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About