Coverage Path Planning Using Reinforcement Learning-Based TSP for hTetran—A Polyabolo-Inspired Self-Reconfigurable Tiling Robot

Abstract

One of the critical challenges in deploying the cleaning robots is the completion of covering the entire area. Current tiling robots for area coverage have fixed forms and are limited to cleaning only certain areas. The reconfigurable system is the creative answer to such an optimal coverage problem. The tiling robot’s goal enables the complete coverage of the entire area by reconfiguring to different shapes according to the area’s needs. In the particular sequencing of navigation, it is essential to have a structure that allows the robot to extend the coverage range while saving energy usage during navigation. This implies that the robot is able to cover larger areas entirely with the least required actions. This paper presents a complete path planning (CPP) for hTetran, a polyabolo tiled robot, based on a TSP-based reinforcement learning optimization. This structure simultaneously produces robot shapes and sequential trajectories whilst maximizing the reward of the trained reinforcement learning (RL) model within the predefined polyabolo-based tileset. To this end, a reinforcement learning-based travel sales problem (TSP) with proximal policy optimization (PPO) algorithm was trained using the complementary learning computation of the TSP sequencing. The reconstructive results of the proposed RL-TSP-based CPP for hTetran were compared in terms of energy and time spent with the conventional tiled hypothetical models that incorporate TSP solved through an evolutionary based ant colony optimization (ACO) approach. The CPP demonstrates an ability to generate an ideal Pareto optima trajectory that enhances the robot’s navigation inside the real environment with the least energy and time spent in the company of conventional techniques.

Reinforcement Learning-Based Path Planning Algorithm for Mobile Robots

“…When V L = V R , the angular velocity of the center of mass of the inspection robot is 0, and then, the inspection robot does linear motion with speed V L or V R ; when V L = −V R , the linear velocity of the center of mass of the inspection robot is 0, and then, the robot can rotate in situ around C [25,26]. The rotation speed is determined by the wheel speed and wheel distance of the inspection robot; when V L ≠ V R and V L ≠ −V R , the robot does circular motion with a certain radius of curvature, and the radius of curvature is calculated as follows.…”

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

“…Another area of research where DRL algorithms were applied is the problem of area coverage. Optimising the coverage path planning is essential for cleaning robots, which are now very popular [35]. The Actor-Critic algorithm based on a convolutional neural network with a long short term memory layer was proved to generate a path with a minimised cost at a lesser time than the genetic algorithms or ant colony optimisation approach [36].…”

Path planning for obstacle avoidance of unmanned vehicles based on Frenet coordinates and B-spline curves

“…Traditional local path planning methods include Bezier curve algorithm [3,9], artificial potential field algorithm [10,11], B-spline curve algorithm [12,13]. In recent years, with the development of artificial intelligence and other technologies, path planning based on reinforcement learning has also attracted many people's attention [14,15]. The main feature of B-spline curve is that the degree can be set by itself, the degree is equal to the degree minus one, and the degree has nothing to do with the number of control points.…”