Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

Abstract

Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of "catastrophic forgetting", in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that employing multiple, complimentary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.The goal of this study was to develop neuroscience-inspired methods to alleviate catastrophic forgetting in ANNs. Two previous studies have proposed one such method: stabilizing connection weights depending on their importance for solving a task [8,9]. This method, inspired by neuroscience research demonstrating that stabilization of dendritic spines is associated with task learning and retention [6,7], has shown promising results when trained and tested on sequences of ≤ 10 tasks. However, it is uncertain how well these methods perform when trained on much larger number of sequential tasks.We first tested whether these methods can alleviate catastrophic forgetting by measuring performance on 100 sequentially presented digit classification tasks. Specifically, we tested a fully connected feedforward network with two hidden layers (2000 units each, Figure 1A) on the permuted MNIST digit classification task [13]. This involved training the network on the MNIST task for 20 epochs, permuting the 784 pixels in all images with the same permutation, and then training the network on this new set of images. This test is a canonical example of an "input reformatting" problem, in which the input and output semantics (pixel intensities and digit identity, respectively) are identical across all tasks, but the input format (the spatial location of each pixel) changes between tasks [13].We sequentially trained the base ANN on 100 permutations of the image set. Without any synaptic stabilization, this network can classify digits with an accuracy of 98.5% for any single permutation, but mean classification accuracy falls to 52.5% after the network is trained on 10 permutation...

How biological agents can couple neural task modules for dealing with the stability-plasticity dilemma

“…It might be suboptimal to always desynchronize all modules that are not currently task-relevant. As suggested by previous work (50), keeping the irrelevant modules at random states (partial gating) might be sufficient to eliminate catastrophic forgetting.…”

“…Because weight change depends on co-activation of relevant neurons (12,69), this approach protects the weights from changing. For example, Masse et al (50) propose that in each of several contexts, a (randomly selected) 80% of nodes is gated out, thus effectively orthogonalizing different contexts. They showed that synaptic gating allowed a multi-layer network to deal with several computationally demanding tasks without catastrophic forgetting.…”

A Neural Model of Schemas and Memory Consolidation

“…Our work is comparable to a recent approach to avoiding catastrophic forgetting by Nakano and Hattori (2017), in which the intermediate layers of a deep neural network are gated by patterns that differ by context. A paper by Masse, Grant, and Freedman (2018) has the similar idea of using CHL as a plausible deep representation of information, and applying "pseudopatterns" alongside their regular training patterns for better separation. Our experiments explain in more depth how these patterns are formed and employed throughout different stages of the learning process.…”

High capacity and dynamic accessibility in associative memory networks with context-dependent neuronal and synaptic gating

“…However, memory capacity is limited in this case, as old memories are quickly overwritten with new ones 36 . Various remedies have been proposed to alleviate forgetting in artificial neural networks during continual learning, including context-dependence and architectural modularity 37,38 , but it is unclear how these methods might operate in a more biologically-realistic setting.…”

“…In this work, we propose a new class of context-dependent associative memory models, which we call context-modular memory networks, inspired by previous theoretical studies 37,45 and experimental findings 10,12,13,21,39,46 . In our model, memories are assigned to different contexts, defined by a set of active neurons and connections.…”