Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments

Abstract

A key challenge for AI is to build embodied systems that operate in dynamically changing environments. Such systems must adapt to changing task contexts and learn continuously. Although standard deep learning systems achieve state of the art results on static benchmarks, they often struggle in dynamic scenarios. In these settings, error signals from multiple contexts can interfere with one another, ultimately leading to a phenomenon known as catastrophic forgetting. In this article we investigate biologically inspired architectures as solutions to these problems. Specifically, we show that the biophysical properties of dendrites and local inhibitory systems enable networks to dynamically restrict and route information in a context-specific manner. Our key contributions are as follows: first, we propose a novel artificial neural network architecture that incorporates active dendrites and sparse representations into the standard deep learning framework. Next, we study the performance of this architecture on two separate benchmarks requiring task-based adaptation: Meta-World, a multi-task reinforcement learning environment where a robotic agent must learn to solve a variety of manipulation tasks simultaneously; and a continual learning benchmark in which the model's prediction task changes throughout training. Analysis on both benchmarks demonstrates the emergence of overlapping but distinct and sparse subnetworks, allowing the system to fluidly learn multiple tasks with minimal forgetting. Our neural implementation marks the first time a single architecture has achieved competitive results in both multi-task and continual learning settings. Our research sheds light on how biological properties of neurons can inform deep learning systems to address dynamic scenarios that are typically impossible for traditional ANNs to solve.

Modelling continual learning in humans with Hebbian context gating and exponentially decaying task signals

“…However, a recent comparison of alternatives suggests that tuning this architecture, specifically using multiplicative forms of gating that act on the output of the non-linearity [ 37 , 62 ], might result in greater task accuracy and support generalisation across tasks [ 52 ]. Multiplicative gating could be implemented by neural oscillations [ 67 ], neurotransmitters [ 62 , 68 ] or even through dendritic properties of neurons [ 69 , 70 ].…”

Thalamo-Cortical Interaction for Incremental Binding in Mental Contour-Tracing

“…On a similar note, it might be useful to further investigate the power of potential binding mechanism of our model in the context of deep learning. For example, within the scope of contextual binding [100] or segmentation challenges [81,23]. There, it also could further be compared to other learned models of incremental binding [21,81,23,22].…”

Dendritic modulation enables multitask representation learning in hierarchical sensory processing pathways

“…One possibility is that fast-spiking inhibitory interneurons target the somata of pyramidal cells (PCs) 36 , effectively opening quasi-tonic conductances in the membrane that decrease input resistance and render it harder for the neuron to emit action potentials (Fig 1C) 37,38 . Another possibility is that modulating inputs target thin dendritic branches 30,39,40 , opening N-Methyl-D-Aspartate (NMDA) channels and eliciting NMDA-spikes 41,42 . These events, whose duration of 50-100 ms outlasts the duration of action potentials by one to two orders of magnitude 43,44 , can also implement a sustained modulation of the neuronal output (Fig 1C).…”

“…Nevertheless, the pervasiveness of contextual modulation in sensory processing indicates that this adaptation is an important component of cortical computation, and reshapes the functional mapping of sensory processing pathways (Fig 1B) 27 . While some authors have explored modulations to early processing layers 28–30 , their networks were trained through error backpropagation in a purely supervised fashion. Unsupervised, representation-based learning is considered more biologically plausible 31–33 , but has not been applied to context-modulated representations.…”