Improved Q-Learning Applied to Dynamic Obstacle Avoidance and Path Planning

Chun-lei, Wang; Han, Qilong; Li., He

doi:10.1109/access.2022.3203072

Cited by 14 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There are different approaches to reinforcement learning techniques, the more common approaches used in path planning are Monte Carlos [64], Q-learning [65], Deep Q Network [66], Twin Delayed Deep Deterministic policy gradient algorithm [67], [68], Deep Deterministic Policy Gradient [69], [70], Soft Actor-Critic [71], Asynchronous Advantage Actor-Critic algorithm [72], [73], Trust Region Policy Optimization [74] and Proximal Policy Optimization (PPO) [72]. Each RL technique offers a performance improvement to the path planning methods.…”

Section: Figure 11 Reinforcement Learning Processmentioning

confidence: 99%

An Overview of Machine Learning Techniques in Local Path Planning for Autonomous Underwater Vehicles

et al. 2023

View full text Add to dashboard Cite

Autonomous underwater vehicles (AUVs) have become attractive and essential for underwater search and exploration because of the advantages they offer over manned underwater vehicles. Hence the need to improve AUV technologies. One crucial area of AUV technology involves efficiently solving the path planning problem. Several approaches have been identified from the literature for AUV global and local path planning. The use of machine learning (ML) techniques in overcoming some of the challenges associated with AUV path planning problems such as safety and obstacle avoidance, energy consumption, and optimal time and distance travelled remains an active research area. While there is literature on global and local path planning that explores different techniques, there is still a lack of paper that provides an overview of the application of ML for local path planning. Hence the main objective of this paper is to present an overview of the state-of-the-art application of ML techniques on local path planning for AUVs. The ML algorithms are discussed under supervised, unsupervised, and reinforcement learning. The challenges faced in real-life deployment, simulated scenarios, computational issues, and application of ML algorithms are discussed, with future research directions presented.INDEX TERMS Machine learning, local path planning, autonomous underwater vehicle (AUV), real-time path planning, underwater.

show abstract

Section: Figure 11 Reinforcement Learning Processmentioning

confidence: 99%

An Overview of Machine Learning Techniques in Local Path Planning for Autonomous Underwater Vehicles

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Further, in the study by Orozco-Rosas et al [13], a new path-planning algorithm with excellent performance was developed by combining Q-learning and the artificial potential field method. Wang et al [19] introduced priority weights into the Q-learning algorithm and applied the algorithm to dynamic obstacle-avoidance path planning.…”

Section: A Path Planningmentioning

confidence: 99%

Generation of Ship’s Passage Plan Using Data-Driven Shortest Path Algorithms

et al. 2022

View full text Add to dashboard Cite

In this study, an approach for generating the shortest-distance passage plan for a ship is proposed considering the navigable area obtained based on automatic identification system data. The navigable area is designated based on the information from the electronic navigational chart and by considering the ship traffic density using Jenks natural breaks classification. The shortest-path algorithms used in our experiment are the Dijkstra, A*, and improved A*. Then, the Douglas-Peucker algorithm is applied to generate an optimal result by supplementing the passage plan generated by the best-performing shortest-path algorithm. The experiment was divided into two cases depending on whether or not the passage plan covered the target area of this study, namely, the vessel traffic service system control area of Busan Port between Busan New Port and Ulsan Port. The improved A* performed better than Dijkstra and A* in these two cases. Subsequently, the distance of the final passage plan was 98.649 km in the case of passing through the control area and 105.365 km in the case of bypassing, which were less than the distances in the passage plan of the actual ship by approximately 13.18% and 7.27%, respectively. The experimental results of the proposed approach show the possibility of automatically establishing the shortest-distance passage plan for a ship considering the navigable area. In addition, this study suggests an effective approach for coastal navigation, which is more complex than ocean navigation. Further, this study can serve as a basis for generating passage plans for maritime autonomous surface ships.

show abstract

“…Some existing algorithms can easily get stuck in local optima when avoiding moving obstacles in complex interactive environments, resulting in defects in convergence speed and accuracy. An improved Q-learning algorithm was proposed, which greatly improves the convergence speed and accuracy of the algorithm and finds better paths in dynamic obstacle path planning [14]. However, as the problem size increases, the Q table in Q-learning algorithms expands, increasing the complexity of the algorithm.…”

Section: Introductionmentioning

confidence: 99%

An Optimal-Path-Planning Method for Unmanned Surface Vehicles Based on a Novel Group Intelligence Algorithm

Chen,

Feng,

Bao

et al. 2024

JMSE

View full text Add to dashboard Cite

Path planning is crucial for unmanned surface vehicles (USVs) to navigate and avoid obstacles efficiently. This study evaluates and contrasts various USV path-planning algorithms, focusing on their effectiveness in dynamic obstacle avoidance, resistance to water currents, and path smoothness. Meanwhile, this research introduces a novel collective intelligence algorithm tailored for two-dimensional environments, integrating dynamic obstacle avoidance and smooth path optimization. The approach tackles the global-path-planning challenge, specifically accounting for moving obstacles and current influences. The algorithm adeptly combines strategies for dynamic obstacle circumvention with an eight-directional current resistance approach, ensuring locally optimal paths that minimize the impact of currents on navigation. Additionally, advanced artificial bee colony algorithms were used during the research process to enhance the method and improve the smoothness of the generated path. Simulation results have verified the superiority of the algorithm in improving the quality of USV path planning. Compared with traditional bee colony algorithms, the improved algorithm increased the length of the optimization path by 8%, shortened the optimization time by 50%, and achieved almost 100% avoidance of dynamic obstacles.

show abstract

Improved Q-Learning Applied to Dynamic Obstacle Avoidance and Path Planning

Cited by 14 publications

References 35 publications

An Overview of Machine Learning Techniques in Local Path Planning for Autonomous Underwater Vehicles

An Overview of Machine Learning Techniques in Local Path Planning for Autonomous Underwater Vehicles

Generation of Ship’s Passage Plan Using Data-Driven Shortest Path Algorithms

An Optimal-Path-Planning Method for Unmanned Surface Vehicles Based on a Novel Group Intelligence Algorithm

Contact Info

Product

Resources

About