Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions

Abstract

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.

Minimal Neural Network Models for Permutation Invariant Agents

“…An early example of a plastic neural network governed by a single learning rule is that of Chalmers [9] that evolves a learning rule to update randomly initialized weights of a shallow neural network. A more recent approach that also falls into this category is presented by Yaman et al [35]. In this work, the authors evolve a single discrete Hebbian rule to change the synapses of a randomly initialized network to solve a simple foraging task.…”

Evolving Hebbian Learning Rules in Voxel-based Soft Robots

“…Past studies [44,45] applied Hebbian learning to evolve ANN controllers for mobile robots, and found improved performance over non-plastic ANNs. More recently, several studies have successfully evolved Hebbian learning rules [46,47,48,49] and achieved competitive results in reinforcement learning scenarios. For these reasons, we adopt Hebbian learning to model synaptic plasticity.…”

“…While several works [59,60,48] have successfully employed this "generalized" Hebbian learning to train ANNs, many variations of it do exist [16]. In this work, we use the socalled Hebbian ABCD model [61,62,63,47], which updates the weights according to: We call the four ABCD coefficients together a rule, and, in our model, there exists one separate rule per synapse.…”

Evolving Decomposed Plasticity Rules for Information-Bottlenecked Meta-Learning

“…For instance, some works neglect the neural modulator m (Najarro & Risi, 2020;Miconi et al, 2018), others have set B, C, and D to 0 (Miconi et al, 2018(Miconi et al, , 2019. The learned rule A, B, C, D will inevitably be dependent on initial parameter of W , however, learning plastic rules that is not dependent on the initial parameters was also investigated (Najarro & Risi, 2020;Yaman et al, 2021).…”

“…In contrast, its parameters scale with O(n 2 ), which makes parameter-updating more powerful as learning mechanisms than recursion-only. Prior to our work, evolving plasticity (Soltoggio et al, 2008(Soltoggio et al, , 2018Lindsey & Litwin-Kumar, 2020;Yaman et al, 2021) has been proposed to reproduce the natural evolution and plasticity in simulation, as shown in Figure 1. Implementing plasticity is not straight-forward, unlike gradient-descent, plastic rules are not universal but have to be optimized beforehand, which is not possible without a further outer-loop optimizer over the inner-loop learning.…”