SLP-Improved DDPG Path-Planning Algorithm for Mobile Robot in Large-Scale Dynamic Environment

Chen, Yinliang; Liang, Liang

doi:10.3390/s23073521

Cited by 13 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of [30] developed a method for path planning using a deterministic policy gradient and a sequential linear path planning algorithm to achieve planning in larger maps, contrary to previous RL limitations.…”

Section: Learning Algorithmsmentioning

confidence: 99%

Path Planning for Autonomous Mobile Robot Using Intelligent Algorithms

Galarza-Falfan,

García-Guerrero,

Aguirre-Castro

et al. 2024

Technologies

View full text Add to dashboard Cite

Machine learning technologies are being integrated into robotic systems faster to enhance their efficacy and adaptability in dynamic environments. The primary goal of this research was to propose a method to develop an Autonomous Mobile Robot (AMR) that integrates Simultaneous Localization and Mapping (SLAM), odometry, and artificial vision based on deep learning (DL). All are executed on a high-performance Jetson Nano embedded system, specifically emphasizing SLAM-based obstacle avoidance and path planning using the Adaptive Monte Carlo Localization (AMCL) algorithm. Two Convolutional Neural Networks (CNNs) were selected due to their proven effectiveness in image and pattern recognition tasks. The ResNet18 and YOLOv3 algorithms facilitate scene perception, enabling the robot to interpret its environment effectively. Both algorithms were implemented for real-time object detection, identifying and classifying objects within the robot’s environment. These algorithms were selected to evaluate their performance metrics, which are critical for real-time applications. A comparative analysis of the proposed DL models focused on enhancing vision systems for autonomous mobile robots. Several simulations and real-world trials were conducted to evaluate the performance and adaptability of these models in navigating complex environments. The proposed vision system with CNN ResNet18 achieved an average accuracy of 98.5%, a precision of 96.91%, a recall of 97%, and an F1-score of 98.5%. However, the YOLOv3 model achieved an average accuracy of 96%, a precision of 96.2%, a recall of 96%, and an F1-score of 95.99%. These results underscore the effectiveness of the proposed intelligent algorithms, robust embedded hardware, and sensors in robotic applications. This study proves that advanced DL algorithms work well in robots and could be used in many fields, such as transportation and assembly. As a consequence of the findings, intelligent systems could be implemented more widely in the operation and development of AMRs.

show abstract

Section: Learning Algorithmsmentioning

confidence: 99%

Path Planning for Autonomous Mobile Robot Using Intelligent Algorithms

Galarza-Falfan,

García-Guerrero,

Aguirre-Castro

et al. 2024

Technologies

View full text Add to dashboard Cite

show abstract

“…Deep reinforcement learning combines the perceptual ability of deep learning and the decision-making ability of reinforcement learning, which effectively solves the problems of dimensional catastrophe and algorithm training in Reinforcement Learning [19], and provides a new idea for solving path-planning problems in complex environments. Deep Q learning [20], Deep Deterministic Policy Gradient (DDPG) [21], Twin Delayed Deep Deterministic Policy Gradient [22], and other methods have been used in the fields of mobile robot task allocation and control optimization. Among them, deep Q-networks are preferred as an improved model for mobile robot path planning due to their good data representation and parameter generalization capabilities.…”

Section: Introductionmentioning

confidence: 99%

“…Where W f is the number of samples in the f-th subset. After many iterations of the clustering center S z is no longer updated to determine the center of the RBF, the width of the RBF function is obtained as shown in equation (21).…”

mentioning

confidence: 99%

Research on mobile robot path planning in complex environment based on DRQN algorithm

Wang,

Du,

Lin

et al. 2024

Phys. Scr.

View full text Add to dashboard Cite

A deep reinforcement Q learning algorithm (DRQN) based on radial neural network is proposed to achieve path planning and obstacle avoidance for mobile robots in complex ground environments with different types of obstacles, including static and dynamic obstacles. Firstly, the path planning problem is represented as a partially-observed Markov decision process. Steering angle, running characteristics, and other elements are introduced into the state-action decision space and the greedy factor is dynamically adjusted using a simulated annealing algorithm, which improves the mobile robot's environment exploration and action selection accuracy. Secondly, the Q-learning algorithm is improved by replacing the Q-table structure with an RBF neural network to enhance the approximation ability of the algorithm's function values, and the parameters of the implicit layer and the weights between the implicit and the output layer are trained using the dynamic clustering and least-mean methods respectively, which improves the convergence speed and enhances the ability of mobile robots to handle large-scale computation. Lastly, the double reward mechanism is set up to prevent the mobile robot from blind searching in unknown environments, which enhances the learning ability and improves path planning safety and flexibility at the same time. Different types of scenarios are set up for simulation experiments, and the results verified the superiority of the DQRN algorithm. Taking the 30 * 30 complex scene as an example, using the DQRN algorithm for path planning reduces the values of distance, turning angle, and planning time by 27.04%, 7.76%, and 28.05%, respectively, compared to the average values of Q-learning, optimized Q-learning, deep Q-learning, and DDPG algorithms, which can effectively improve the path planning efficiency for mobile robots in complex environments.

show abstract

“…Junjie Zeng et al [ 11 ] successfully guided a robot’s moves through continuous control signals in a dynamic environment by combining the Jump Point Search (JPS) algorithm with the asynchronous advantage Actor–Critic (A3C) algorithm. Yinliang Chen et al [ 12 ] proposed an improved deep deterministic policy gradient (DDPG) path planning algorithm incorporating sequential linear path planning (SLP) to address the problem of robot avoidance of dynamic obstacles. In 2020, MG et al [ 13 ] investigated the problem of multi-intelligence collaboration.…”

Section: Introductionmentioning

confidence: 99%

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

Han

Wang

Kuang

et al. 2023

Sensors

View full text Add to dashboard Cite

With the advancement of robotics, the field of path planning is currently experiencing a period of prosperity. Researchers strive to address this nonlinear problem and have achieved remarkable results through the implementation of the Deep Reinforcement Learning (DRL) algorithm DQN (Deep Q-Network). However, persistent challenges remain, including the curse of dimensionality, difficulties of model convergence and sparsity in rewards. To tackle these problems, this paper proposes an enhanced DDQN (Double DQN) path planning approach, in which the information after dimensionality reduction is fed into a two-branch network that incorporates expert knowledge and an optimized reward function to guide the training process. The data generated during the training phase are initially discretized into corresponding low-dimensional spaces. An “expert experience” module is introduced to facilitate the model’s early-stage training acceleration in the Epsilon–Greedy algorithm. To tackle navigation and obstacle avoidance separately, a dual-branch network structure is presented. We further optimize the reward function enabling intelligent agents to receive prompt feedback from the environment after performing each action. Experiments conducted in both virtual and real-world environments have demonstrated that the enhanced algorithm can accelerate model convergence, improve training stability and generate a smooth, shorter and collision-free path.

show abstract

SLP-Improved DDPG Path-Planning Algorithm for Mobile Robot in Large-Scale Dynamic Environment

Cited by 13 publications

References 24 publications

Path Planning for Autonomous Mobile Robot Using Intelligent Algorithms

Path Planning for Autonomous Mobile Robot Using Intelligent Algorithms

Research on mobile robot path planning in complex environment based on DRQN algorithm

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

Contact Info

Product

Resources

About