CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks

Abstract

This paper studies continual learning (CL) of a sequence of aspect sentiment classification (ASC) tasks in a particular CL setting called domain incremental learning (DIL). Each task is from a different domain or product. The DIL setting is particularly suited to ASC because in testing the system needs not know the task/domain to which the test data belongs. To our knowledge, this setting has not been studied before for ASC. This paper proposes a novel model called CLASSIC. The key novelty is a contrastive continual learning method that enables both knowledge transfer across tasks and knowledge distillation from old tasks to the new task, which eliminates the need for task ids in testing. Experimental results show the high effectiveness of CLASSIC. 1 * Work was done prior to joining Amazon.

Continuous Detection, Rapidly React: Unseen Rumors Detection based on Continual Prompt-Tuning

“…The prompt-based model is better than the adapter in both datasets. Consistent with Liu et al (2021b), p-tuning v2 uses CLS token classification output better than using verbalizer under multi-task learning. The performance of multi-task learning is usually considered the upper limit of the CL model.…”

“…Instead of searching for discrete template words, Li and Liang (2021) propose prefix-tuning, where tokens with trainable continuous embeddings are placed at the beginning of the text to perform generate tasks. Ptuning v2 (Liu et al, 2021b) also uses soft-prompt to achieve promising natural language understanding and knowledge probing tasks. Different from the above methods, they studied single-step adaptation, and we are interested in prompt transfer in CL environment.…”

“…(2) Knowledge transfer and accumulation: Transfer the knowledge learned from upstream tasks for rapid generalization, accumulating knowledge from downstream tasks to better cope with upstream tasks; (3) Parameter explosion: Previous research (Wang et al, 2021a;Ke et al, 2021a) often require dynamically expanding neural modules for each task, which is undoubtedly aggravating for pre-trained language models (PLMs) with billions of parameters, with limited memory; (4) Data privacy: After learning a task, its training data is often discarded (Chen and Liu, 2018). Data owners do not want their data to be used by others when the model needs to share the learned parameters, not the data (Ke et al, 2021b).…”

“…2) to increase task-specific parameters. Technically, CPT-RD can be viewed as a continual learning version of P-tuning v2 (Liu et al, 2021b). The most significant improvements come from the storage memory of task-specific parameters and bidirectional domain knowledge transfer, as follows: (1) Forward Knowledge Transfer: CPT-RD has various prompt initialization strategies to adapt fast to rumors of emergencies.…”

SentiStream: A Co-Training Framework for Adaptive Online Sentiment Analysis in Evolving Data Streams

“…In a noteworthy contribution, Jin et al (2021) deployed distillationbased techniques for the continuous incremental pre-training of language models across diverse domain corpora. In the context of sentiment analysis, Ke et al (2021) explored aspect-based sentiment analysis tasks across different domains through contrastive continual learning. However, these approaches often fail to address the temporal influence, as observed by Luu et al (2021), where data drift over time can negatively impact model performance.…”