Remembering for the right reasons: Explanations reduce catastrophic forgetting

Abstract

The goal of continual learning (CL) is to learn a sequence of tasks without suffering from the phenomenon of catastrophic forgetting. Previous work has shown that leveraging memory in the form of a replay buffer can reduce performance degradation on prior tasks. We hypothesize that forgetting can be further reduced when the model is encouraged to remember the evidence for previously made decisions. As a first step towards exploring this hypothesis, we propose a simple novel training paradigm, called Remembering for the Right Reasons (RRR), that additionally stores visual model explanations for each example in the buffer and ensures the model has "the right reasons" for its predictions by encouraging its explanations to remain consistent with those used to make decisions at training time. Without this constraint, there is a drift in explanations and increase in forgetting as conventional continual learning algorithms learn new tasks. We demonstrate how RRR can be easily added to any memory or regularization-based approach and results in reduced forgetting, and more importantly, improved model explanations. We have evaluated our approach in the standard and fewshot settings and observed a consistent improvement across various CL approaches using different architectures and techniques to generate model explanations and demonstrated our approach showing a promising connection between explainability and continual learning. Our code is available at https://github.com/ SaynaEbrahimi/Remembering-for-the-Right-Reasons.

Saliency Guided Experience Packing for Replay in Continual Learning

“…Since later layers in the convolutional neural network capture high-level semantics (Mahendran and Vedaldi 2016), taking gradients of a model output with respect to the feature map activations from one such layers identifies which high-level semantics are important for the model prediction. In our analysis, we select this layer and refer to as target layer (Ebrahimi et al 2021). List of target layer for different experiments is given in Table 3.…”

“…Guo et al (2020) improved such methods by proposing a loss balancing update rules in MEGA. Ebrahimi et al (2021) stores a saliency map corresponding to each episodic memory sample and complements replay with a regularization objective so that model explanations for the past tasks have minimal drift. Our method, EPR, also uses episodic memory for experience replay.…”

“…Once the memory patches are selected and stored in M E , the temporary memory, M T is freed up to be reused in the next task. If we assume that data from k-th task is available to the model until it sees the next tasks (as in Ebrahimi et al (2021)), M T is not needed.…”

BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Belief

“…Finally, our system performs a kind of continual learning (Parisi et al, 2019;Carlson et al, 2010). While recent work in this area has focused on dynamic update of model parameters, e.g., (Ebrahimi et al, 2021), our work leaves the model fixed, and seeks improvement in the broader system in which the model is embedded, exploring an alternative and potentially more interpretable architecture towards this goal.…”

Optimizing Reusable Knowledge for Continual Learning via Metalearning

“…Previous works have tackled the problem of CF using different strategies. Memory-based methods mitigate CF inserting data from past tasks into the training process of new tasks, continuously retraining previous tasks [30], either with raw samples [26,31], or minimizing gradient interference [28,32]. Other works such as [33,34] train generator functions (GANs) or autoencoders [35] to Similarly, [36,37] seek to be memory-efficient by saving feature vectors of instances from previous tasks, while learning a transformation from the feature space of past tasks to current ones.…”