Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning

Abstract

Continual learning (CL) learns a sequence of tasks incrementally with the goal of achieving two main objectives: overcoming catastrophic forgetting (CF) and encouraging knowledge transfer (KT) across tasks. However, most existing techniques focus only on overcoming CF and have no mechanism to encourage KT, and thus do not do well in KT. Although several papers have tried to deal with both CF and KT, our experiments show that they suffer from serious CF when the tasks do not have much shared knowledge. Another observation is that most current CL methods do not use pre-trained models, but it has been shown that such models can significantly improve the end task performance. For example, in natural language processing, fine-tuning a BERT-like pre-trained language model is one of the most effective approaches. However, for CL, this approach suffers from serious CF. An interesting question is how to make the best use of pre-trained models for CL. This paper proposes a novel model called CTR to solve these problems. Our experimental results demonstrate the effectiveness of CTR. 2 * Work was done prior to joining Amazon. 2 The code of CTR can be found at https://github.com/ZixuanKe/PyContinual 35th Conference on Neural Information Processing Systems (NeurIPS 2021),

Contrastive Learning for Boosting Knowledge Transfer in Task-Incremental Continual Learning of Aspect Sentiment Classification Tasks

“…Recent years have seen a surge of research efforts focusing on the integration of advanced pre-trained models, such as BERT (Bidirectional Encoder Representations from Transformers) [5], into continual learning approaches. BERT adapters, lightweight task-specific neural network components, have emerged as a promising solution [9], which chooses to freeze the parameters of the pre-trained BERT, only tuning a small number of parameters of injected taskspecific adapters [9][10][11]. A BERT adapter can be a relatively small 2-layer fully connected network or a Capsule Networks (CapsNet) [12,13] that uses vector capsules instead of scalar feature detectors.…”

“…Although the CL capability of existing advanced models have been empirically demonstrated they still struggle with CF issues when tasks lack substantial shared knowledge [11,14].…”

“…As far as we know, only a handful of continual learning (CL) models have effectively achieved both objectives simultaneously. Notably, CTR [11] and CLASSIC [14] are two prominent CL models that have recently demonstrated success in this regard. Both models employ task masks to isolate task-specific knowledge for dealing with CF.…”

“…When learning a new task, only the module specific for this task is tuned while others remain untouched. These modules can be task-specific adapters [9,10] or continual learning plugins (CL-plugin) [11] or a relatively small 2-layer fully connected network (so-called a Capsule Networks) [12,13], which are injected into the pre-trained BERT.…”

“…To our knowledge, only a few proposed CL models successfully achieve those two objectives at the same time. Among them, CTR [11] and CLASSIC [14] recently proposed are two wellestablished CL models. The former belongs to the TIL setting while the latter is of the DIL setting.…”