Generating Collective Behavior of a Multi-Legged Robotic Swarm Using Deep Reinforcement Learning

Abstract

This paper presents a method of generating collective behavior of a multi-legged robotic swarm using deep reinforcement learning. Most studies in swarm robotics have used mobile robots driven by wheels. These robots can operate only on relatively flat surfaces. In this study, a multi-legged robotic swarm was employed to generate collective behavior not only on a flat field but also on rough terrain fields. However, designing a controller for a multi-legged robotic swarm becomes a challenging problem because it has a large number of actuators than wheeled-mobile robots. This paper applied deep reinforcement learning to designing a controller. The proximal policy optimization (PPO) algorithm was utilized to train the robot controller. The controller was trained through the task that required robots to walk and form a line. The results of computer simulations showed that the PPO led to the successful design of controllers for a multi-legged robotic swarm in flat and rough terrains.

AI Driven Approaches in Swarm Robotics - A review

“…The paper [52] introduces a novel approach to generating collective behavior in multi-legged robotic swarms. focuses on employing the Proximal Policy Optimization (PPO) algorithm to generate collective behavior in a multi-legged underwater is possible and their methods effectively maintained network integrity while efficiently transferring energy for the robots' tasks.…”

“…This concise overview facilitates a comprehensive understanding of the diverse applications of AI in swarm robotics, alongside the testing environments and specific methodologies employed across the studies. -√ Large language model (LLM) [21] -√ RL algorithm [5] √ -Dueling Double Deep Q-Network (D3QN) [6] √ -Deep Learning Trained by Genetic Algorithm (DL-GA) [8] √ -3D StringNet herding [10] √ -Decision-making mechanisms [12] √ -Deep Imitation Reinforcement Learning (DIRL) [17] Augmented Lagrangian particle swarm optimization (ALPSO) [20] √ √ Automatic modular design approach (AutoMoDe) [24] Coordination -√ AudioLocNetv(deep learning module) [31] √ -Not specified [32] √ -End-to-end Neural Networks to train robots [27] √ -Mean-field feedback control [28] √ -Deep Neural Network (DNN) model [29] √ -variant of the crawling probabilistic road map motion planning algorithm [33] √ -distributed online reinforcement learning method [34] √ -coordination algorithm [51] Optimization -√ PSO algorithm [53] -√ streamlined algorithms [36] √ -Genetic algorithm (GA) [46] √ -Particle Swarm Optimization (PSO) [49] √ -Robot Bean Optimization Algorithm (RBOA) [50] √ -Automatic modular design method: AutoMoDe-Cedrata and AutoMoDe-Maple [52] √ -PPO algorithm [54] √ -Dijkstra algorithm [55] √ -WC and WET algorithms [44] √ √ Decentralized ergodic planning [35] Optimization and Navigation √ -YOLOv8 [41] √ -Quantum-based path-planning algorithm and Grover's search algorithm [42] √ -Genetic algorithms (GA) and Cellular automata techniques [9] √ -Mean-Field Control (MFC), deep reinforcement learning (RL), and collision avoidance algorithms [22] Optimization and Coordination √ -Knowledge-Based Neural Ordinary Differential Equations (KNODE) [23] √ -Surrogate models ...…”