ImprovedQ‐Learning Method for Multirobot Formation and Path Planning with Concave Obstacles

Abstract

Aiming at the formation and path planning of multirobot systems in an unknown environment, a path planning method for multirobot formation based on improved Q -learning is proposed. Based on the leader-following approach, the leader robot uses an improved Q -learning algorithm to plan the path and the follower robot achieves a tracking strategy of gravitational potential field (GPF) by designing a cost function to select actions. Specifically, to improve the Q-learning, Q -value is initialized by environmental guidance of the target’s GPF. Then, the virtual obstacle-filling avoidance strategy is presented to fill non-obstacles which is judged to tend to concave obstacles with virtual obstacles. Besides, the simulated annealing (SA) algorithm whose controlling temperature is adjusted in real time according to the learning situation of the Q -learning is applied to improve the action selection strategy. The experimental results show that the improved Q -learning algorithm reduces the convergence time by 89.9% and the number of convergence rounds by 63.4% compared with the traditional algorithm. With the help of the method, multiple robots have a clear division of labor and quickly plan a globally optimized formation path in a completely unknown environment.

[Retracted] Research on Tourism Route Planning Based on Artificial Intelligence Technology

“…With the help of artificial intelligence, all fields of society have shown a broad development space, which not only changes the development mode of various fields but also changes the overall framework, development concept, and operation mode of various fields. Among them, artificial intelligence is widely used in the field of education, which has also attracted the attention of universities and many researchers [4].…”

Research on mobile robot path planning in complex environment based on DRQN algorithm

“…When the value of e is closer to 1, the mobile robot tends to explore the environment; when the value of e is closer to 0, the mobile robot tends to use the action with the largest Q value in the current environment. Setting the e factor, This strategy may fall into the local optimum and have low convergence efficiency, so a simulated annealing algorithm [27] is introduced to dynamically adjust the e-factor. The algorithm jumps out of the local optimal solution by searching for the difference between the front and back energy produced by the solid temperature change during the iteration process.…”

“…Due to the observation space set in the algorithm, more environmental information needs to be collected in a shorter time to ensure the safety of the planned path. The euclidean distance between two action nodes in the environment space is determined to enable the robot to obtain well-directed environmental information, and computational relationship between distance and reward is shown in equation (27). where (x , t y t ) and ( + x , t 1 + y t 1 ) are the coordinates of the robot's movement nodes at moments t and t + 1, respectively.…”