Learning to Move in Modular Robots using Central Pattern Generators and Online Optimization

Abstract

This article addresses the problem of how modular robotics systems, i.e. systems composed of multiple modules that can be configured into different robotic structures, can learn to locomote. In particular, we tackle the problems of online learning, that is, learning while moving, and the problem of dealing with unknown arbitrary robotic structures.We propose a framework for learning locomotion controllers based on two components: a central pattern generator (CPG) and a gradient-free optimization algorithm: Powell's method. The CPG is implemented as a system of coupled nonlinear oscillators in our YaMoR modular robotic system, with one oscillator per module. The nonlinear oscillators are coupled together across modules using Bluetooth communication to obtain specific gaits, i.e. synchronized patterns of oscillations among modules. Online learning is done by running the Powell optimization algorithm in parallel to the CPG model, with the speed of locomotion being the criterion to be optimized. Interesting aspects of the optimization are: it is carried out online, it does not require stopping or resetting the robots, and it is fast.We present results showing the interesting properties of this framework for a modular robotic system. In particular, our CPG model can readily be implemented in a distributed system, it is cheap computationally, it exhibits limit cycle behavior (temporary perturbations are rapidly forgotten), it produces smooth trajectories even when control parameters are abruptly changed, and it is robust against imperfect communication among modules. We also present results of learning to move with three different robot structures. Interesting locomotion modes are obtained after running the optimization for less than 60 minutes.

Adaptive creeping locomotion of a CPG-controlled snake-like robot to environment change

“…Based on biomimetic central pattern generators and on information from distributed distance sensors, neuromuscular motion control for an undulatory robot models was presented (Sfakiotakis and Tsakiris 2008). By implementing the CPG as a system of coupled nonlinear oscillators in the modular robotic system, Sproewitz ad-dressed the problem of learning to locomote in modular robotic systems which composed of multiple modules that can be configured into different robotic structures (Sproewitz et al 2008).…”

Adaptive strategy for online gait learning evaluated on the polymorphic robotic LocoKit

“…Parameters were manually extracted from the modular robot by exploiting symmetries. Follow-up work by Spröwitz et al demonstrated online optimization of 6 parameters on a physical robot in roughly 25-40 minutes [21]. We also try to realize simple, robust, fast, model-free, life-long learning on a modular robot.…”

Resilient Robots: Concept, Review, and Future Directions

“…Most robots learn their behaviors by tuning controllers [27], and some of them can adapt to unexpected situations [24]. The work in [28] used central-pattern-generator-based strategy to find efficient locomotion gaits after failures of several actuators.…”