Quality-Oriented Hybrid Path Planning Based on A* and Q-Learning for Unmanned Aerial Vehicle

Abstract

Unmanned aerial vehicles (UAVs) are playing an increasingly important role in people's daily lives due to their low cost of operation, low requirements for ground support, high maneuverability, high environmental adaptability, and high safety. Yet UAV path planning under various safety risks, such as crash and collision, is not an easy task, due to the complicated and dynamic nature of path environments. Therefore, developing an efficient and flexible algorithm for UAV path planning has become inevitable. Aimed at quality-oriented UAV path planning, this paper is designed to analyze UAV path planning from two aspects: global static planning and local dynamic hierarchical planning. Through a theoretical and mathematical approach, a three-dimensional UAV path planning model was established. Based on the A* algorithm, the search strategy, the step size, and the cost function were improved, and the OPEN set was simplified, thereby shortening the planning time and greatly improving the execution efficiency of the algorithm. Moreover, a dynamic exploration factor was added to the exploration mechanism of Q-learning to solve the explorationexploitation dilemma of Q-learning to adapt to the local dynamic path adjustment for UAVs. The global-local hybrid UAV path planning algorithm was formed by combining the two. The simulation results indicate that the proposed planning model and algorithm can efficiently solve the problem of UAV path planning, improve the path quality, and can be a significant reference for solving other problems related to path planning, such as the reliability, security, and safety of UAV, when embedded into the heuristic function of the proposed algorithm.

Multi‐UAV path planning utilizing the PGA algorithm for terrestrial IoT sensor network under ISAC framework

“…Nevertheless, this approach, unlike probabilistic methodologies that offer greater adaptability and flexibility, necessitates heightened knowledge and time for generation. A comprehensive evaluation of UAV‐based value‐added IoT services is featured in Reference 16, centering on UAV collision avoidance strategies, challenges in wireless communication within flying ad‐hoc networks (FANET), the utilization of on‐board UAV sensors, and the processing of pertinent data amassed by UAVs.…”

Path Planning Research of a UAV Base Station Searching for Disaster Victims’ Location Information Based on Deep Reinforcement Learning

“…The execution architecture of the DDQN-SSQN algorithm is shown in Figure 5 . The action policy used by the algorithm is the -greedy policy [ 38 ], i.e., the Agent has a probability of executing the action corresponding to the maximum Q function in the scene model with . The execution action A can be expressed as: …”

Static and Dynamic Path Planning Algorithms Design for a Wheeled Mobile Robot Based on a Hybrid Technique

“…walls, doors, furniture, etc.) [14,15]. This paper is organized as follows: Section 2 addresses kinematics models of differential wheeled mobile robots.…”

“…Figure. 23 Movement of the obstacle in the environment5.2 The second scenario (dynamic environment)The second scenario in the dynamic environment used moveable obstacles in the workspace depending on the proposed equations as shown below:𝑥 𝑝(𝑛𝑒𝑤) = 𝑥 𝑝(𝑜𝑙𝑑) + 𝑉𝑜𝑏𝑠 × 𝑐𝑜𝑠Ɵ × 𝑇𝑠 (14)𝑦 𝑝(𝑛𝑒𝑤) = 𝑦 𝑝(𝑜𝑙𝑑) + 𝑉𝑜𝑏𝑠 × 𝑠𝑖𝑛Ɵ × 𝑇𝑠(15) …”