A Path-Planning Method Based on Improved Soft Actor-Critic Algorithm for Mobile Robots

Abstract

The path planning problem has gained more attention due to the gradual popularization of mobile robots. The utilization of reinforcement learning techniques facilitates the ability of mobile robots to successfully navigate through an environment containing obstacles and effectively plan their path. This is achieved by the robots’ interaction with the environment, even in situations when the environment is unfamiliar. Consequently, we provide a refined deep reinforcement learning algorithm that builds upon the soft actor-critic (SAC) algorithm, incorporating the concept of maximum entropy for the purpose of path planning. The objective of this strategy is to mitigate the constraints inherent in conventional reinforcement learning, enhance the efficacy of the learning process, and accommodate intricate situations. In the context of reinforcement learning, two significant issues arise: inadequate incentives and inefficient sample use during the training phase. To address these challenges, the hindsight experience replay (HER) mechanism has been presented as a potential solution. The HER mechanism aims to enhance algorithm performance by effectively reusing past experiences. Through the utilization of simulation studies, it can be demonstrated that the enhanced algorithm exhibits superior performance in comparison with the pre-existing method.

Research on mobile robot path planning in complex environment based on DRQN algorithm

“…The number of input layers and the quantitative relationship is G = T + 1. The input vector is shown in equation (18). The second layer of the RBF network is the hidden layer with s nodes and the basis function is j k (Y) (k = 1, 2, L, T, G).…”

“…Taking the CJ8 environment as an example, when the mobile robot traveled to the coordinate points of (5, 25), (7,15), (18,14), and (24, 10), it was observed that there was a dynamic obstacle in the vicinity that changed its state, and according to this paper's algorithm, it changed the direction of the mobile robot's operation in advance, chose the next action and coordinates, and then changed the traveling route to successfully avoid obstacles and continue the path planning. It is proved that the practicality of this paper's algorithm is good.…”

“…Xue et al [17] combined the velocity Mutually Reversible Obstacle concept with the Proximal Policy Optimization reinforcement algorithm to improve the intelligent body action space and reward function, mapping the state of obstacles to action selection, and obtaining a reasonable collision avoidance path for unmanned boats. Zhao et al [18] combined the concept of Maximum Entropy in the reinforcement learning Soft Actor-Critic algorithm to enhance sample usage in the training phase and performed path planning in unknown environments through the experience playback mechanism. In the above studies, the optimal policy of the value-based reinforcement learning algorithm represented by the Q-learning algorithm remains unchanged, but the discretization of the values in the continuous action space produces a dimensional catastrophe, which leads to slow convergence.…”