Evolving Hebbian Learning Rules in Voxel-based Soft Robots

Abstract

<div>According to Hebbian theory, synaptic plasticity is the ability of neurons to strengthen or weaken the synapses among them in response to stimuli. It plays a fundamental role in the processes of learning and memory of biological neural networks. With plasticity, biological agents can adapt on multiple timescales and outclass artificial agents, the majority of which still rely on static Artificial Neural Network (ANN) controllers. In this work, we focus on Voxel-based Soft Robots (VSRs), a class of simulated artificial agents, composed as aggregations of elastic cubic blocks. We propose a Hebbian ANN controller where every synapse is associated with a Hebbian rule that controls the way the weight is adapted during the VSR lifetime. For a given task and morphology, we optimize the controller for the task of locomotion by evolving, rather than the weights, the parameters of the Hebbian rules. Our results show that the Hebbian controller is comparable, often better than a non-Hebbian baseline and that it is more adaptable to unforeseen damages. We also provide novel insights into the inner workings of plasticity and demonstrate that "true" learning does take place, as the evolved controllers improve over the lifetime and generalize well.</div>

Modular Controllers Facilitate the Co-Optimization of Morphology and Control in Soft Robots

“…Soft Robotics The field of soft robots with volumetric actuation started with [19,21,55] and with the availability of simulators such as [3,20,32,35], many others have followed. Soft robots are evolved for locomotion tasks in different environments [8,11,24,28], their ability to change their shape volumetrically are investigated [5,29,50], different types of control strategies are developed [12,17,34,43]. Lifetime development in a co-optimization setting is studied in [10,26,27].…”

“…It consists of a mass-spring system-based soft-body simulation engine and various task environments. Similar to the simulation engines in [17,24,[34][35][36][37]43], EvoGym works in 2D. The simulation engine and the provided environments are open-source and provide Python API for fast prototyping and experimenting.…”

“…Global Control Similar to [17,22,34], we devise a straightforward control for the robots where a centralized, global controller processes the complete information of the soft robot to assign actions to each voxel, as illustrated in Figure 2a. Observations coming from each voxel are concatenated into vector 𝑂, and the global controller 𝑓 𝐺 processes all the available information to output vector 𝐴 that contains actions for each voxel as in 𝑓 𝐺 (𝑂) = 𝐴.…”

“…Following the common practice of utilizing neural networks as powerful function approximators [17,22,34,36,37,43,44,[52][53][54], the controllers 𝑓 𝑀 and 𝑓 𝐺 are modeled by a single hidden layer MLP with learnable parameters 𝜃 𝑀 and 𝜃 𝐺 , respectively. The hidden layer consists of 32 units with ReLU activations for both controllers and maps the observations into a feature vector.…”

Evolving Modular Soft Robots without Explicit Inter-Module Communication using Local Self-Attention

“…Attention can be seen as an "adaptive weights" [19] mechanism that computes importance scores for the inputs. Attention mechanisms were first introduced in the context of machine translation [2,45] to capture relationships in temporal sequences of data, and have thus prospered in natural language processing [13,21].…”