Reinforcement Learning-Based Complete Area Coverage Path Planning for a Modified hTrihex Robot

Abstract

One of the essential attributes of a cleaning robot is to achieve complete area coverage. Current commercial indoor cleaning robots have fixed morphology and are restricted to clean only specific areas in a house. The results of maximum area coverage are sub-optimal in this case. Tiling robots are innovative solutions for such a coverage problem. These new kinds of robots can be deployed in the cases of cleaning, painting, maintenance, and inspection, which require complete area coverage. Tiling robots’ objective is to cover the entire area by reconfiguring to different shapes as per the area requirements. In this context, it is vital to have a framework that enables the robot to maximize the area coverage while minimizing energy consumption. That means it is necessary for the robot to cover the maximum area with the least number of shape reconfigurations possible. The current paper proposes a complete area coverage planning module for the modified hTrihex, a honeycomb-shaped tiling robot, based on the deep reinforcement learning technique. This framework simultaneously generates the tiling shapes and the trajectory with minimum overall cost. In this regard, a convolutional neural network (CNN) with long short term memory (LSTM) layer was trained using the actor-critic experience replay (ACER) reinforcement learning algorithm. The simulation results obtained from the current implementation were compared against the results that were generated through traditional tiling theory models that included zigzag, spiral, and greedy search schemes. The model presented in the current paper was also compared against other methods where this problem was considered as a traveling salesman problem (TSP) solved through genetic algorithm (GA) and ant colony optimization (ACO) approaches. Our proposed scheme generates a path with a minimized cost at a lesser time.

Application of PID Control and Improved Ant Colony Algorithm in Path Planning of Substation Inspection Robot

“…In the substation design, the roads in feasible areas are mostly flat. However, the substation roads in mountainous areas might have potholes, and there will be slopes or steps in places with large drops [15]. e inspection robot needs to regularly conduct comprehensive and detailed inspections on the power equipment in the substation, greatly improving the efficiency and accuracy of inspection.…”

Deep Reinforcement Learning Algorithms for Path Planning Domain in Grid-like Environment

“…Optimising the coverage path planning is essential for cleaning robots, which are now very popular [35]. The Actor-Critic algorithm based on a convolutional neural network with a long short term memory layer was proved to generate a path with a minimised cost at a lesser time than the genetic algorithms or ant colony optimisation approach [36]. The recurrent neural network with long short term memory layers using DRL was proposed to solve the coverage path planning problem in [37].…”

Modelling and Control of a Reconfigurable Robot for Achieving Reconfiguration and Locomotion with Different Shapes

“…Although the above-mentioned robots are reconfigurable, the reconfiguration is limited to a small set of predefined shapes (e.g., hTetrakis: three shapes [ 21 ], hTetro: seven shapes [ 19 , 25 , 26 ], hHoneycomb: seven shapes [ 20 ], and hTrihex: two shapes [ 27 ]). According to [ 28 , 29 , 30 ], area coverage performance of a reconfigurable robot can be improved by considering beyond a small set of predefined shapes for the reconfiguration.…”

“…The majority of floor cleaning reconfigurable robots such as hTetrakis [ 21 ], hHoneycomb [ 20 ], hTrihex [ 27 ], and initial versions of hTetro [ 19 ] have active hinges for reconfiguration. The reconfiguration of the robots with active hinges is carried out through the operation of the servo motors attached to the hinges, and no kinematic for drive mechanisms is utilized for the reconfiguration.…”