Optimizing Reusable Knowledge for Continual Learning via Metalearning

Abstract

When learning tasks over time, artificial neural networks suffer from a problem known as Catastrophic Forgetting (CF). This happens when the weights of a network are overwritten during the training of a new task causing forgetting of old information. To address this issue, we propose MetA Reusable Knowledge or MARK, a new method that fosters weight reusability instead of overwriting when learning a new task. Specifically, MARK keeps a set of shared weights among tasks. We envision these shared weights as a common Knowledge Base (KB) that is not only used to learn new tasks, but also enriched with new knowledge as the model learns new tasks. Key components behind MARK are two-fold. On the one hand, a metalearning approach provides the key mechanism to incrementally enrich the KB with new knowledge and to foster weight reusability among tasks. On the other hand, a set of trainable masks provides the key mechanism to selectively choose from the KB relevant weights to solve each task. By using MARK, we achieve state of the art results in several popular benchmarks, surpassing the best performing methods in terms of average accuracy by over 10% on the 20-Split-MiniImageNet dataset, while achieving almost zero forgetfulness using 55% of the number of parameters. Furthermore, an ablation study provides evidence that, indeed, MARK is learning reusable knowledge that is selectively used by each task.Unfortunately, in the case of ANNs, task interference is more severe than in the case of humans. This is evident for the case of continual learning, i.e., the case when data from new tasks is presented sequentially to the learner, who does not have access to previous data. Under this training scheme, when a previously trained model is retrained on a new task, it usually suffers a significant drop in performance in the original task [8,9,10]. Previous works to prevent CF have followed two main strategies. The first strategy [8,10,11,12,13] consists of avoiding the modification of parameters Preprint. Under review.

Recent Advances of Continual Learning in Computer Vision: An Overview

“…The neuromodulatory neural network is meta-trained to activate only a subset of parameters of the prediction learning network for each task in order to mitigate catastrophic forgetting. Hurtado et al [167] proposed a method called Meta Reusable Knowledge (MARK), which has a single common knowledge base for all the learned tasks. For each new task, MARK uses meta learning to update the common knowledge base and uses a trainable mask to extract the relevant parameters from the knowledge base for the task.…”