Not Just Selection, but Exploration: Online Class-Incremental Continual Learning via Dual View Consistency

Gu, Yanan; Xu, Yang; Wei, Kun; Deng, Cheng

doi:10.1109/cvpr52688.2022.00729

Cited by 45 publications

(32 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Replay-based methods [11,20] preserve examples of fixed memory size to maintain the distribution of old classes in the incremental phases. Selection and retrieval [3,14,40] strategies for the buffer are also developed to preserve decision boundaries. Recently, some practical yet challenging settings of class-incremental learning, including Online Class-Incremental continual Learning (OCL), Non-Exemplar Class-incremental continual Learning (NE-CL), and Few-Shot Class-incremental Learning (FS-CL) are also proposed.…”

Section: Related Workmentioning

confidence: 99%

“…Online Class-incremental continual Learning (OCL) OCL aims to learn new classes continually from online data streams (each sample is seen only once). As the model needs to learn novel classes from the data stream while not forgetting previous classes, OCL methods [34,5,3,35,15,14,50] follow replay-based protocols, where example buffers are stored and retrieved between data batches and tasks. Concretely, [15,14] dig the critical information by maximizing their mutual information.…”

Section: Related Workmentioning

confidence: 99%

“…In privacy-sensitive online scenarios, a practical Online Class-incremental Continual Learning (OCL) system is ex- pected to learn novel classes incrementally while keeping the prior knowledge without restoring any streaming data due to the privacy concerns and computation resource constraints. Existing OCL methods [15,40,14,35,50] mainly focus on exploring the critical feature representation [15,35,14,50] and developing strategies of selecting and retrieving proper samples from the data buffer [40,14] to re-calibrate the decision boundaries. However, these methods have two obvious drawbacks.…”

Section: Introductionmentioning

confidence: 99%

“…However, existing OCL solutions heavily rely on the example buffer for replay between data batches and tasks [12]. Secondly, OCL mainly concerns the setting of online training from scratch, and the state-of-the-art method [14] achieves relatively low accuracy (<20% for CIFAR100 [28] with 1000 buffer size), which significantly limits the OCL application to practical online scenarios.…”

Section: Introductionmentioning

confidence: 99%

“…Class-wise accuracy, online training time, and memory overhead comparisons on the CIFAR100 dataset [28] with the same protocols of O 2 CL. Stat-of-the-art OCL (SCR [35], DVC [14], and OCM [15], all with 1000 example buffer), FS-CL (ALICE [38]), and NE-CL (SSRE [54]) methods are illustrated.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Offline-Online Class-incremental Continual Learning via Dual-prototype Self-augment and Refinement

Huo¹,

Xu²,

Guo³

et al. 2023

Preprint

View full text Add to dashboard Cite

This paper investigates a new, practical, but challenging problem named Offline-Online Class-incremental Continual Learning (O 2 CL), which aims to preserve the discernibility of pre-trained (i.e., offline) base classes without buffering data examples, and efficiently learn novel classes continuously in a single-pass (i.e., online) data stream. The challenges of this task are mainly two-fold: 1) Both base and novel classes suffer from severe catastrophic forgetting as no previous samples are available for replay. 2) As the online data can only be observed once, there is no way to fully re-train the whole model, e.g., re-calibrate the decision boundaries via prototype alignment or feature distillation. In this paper, we propose a novel Dual-prototype Selfaugment and Refinement method (DSR) for O 2 CL problem, which consists of two strategies: 1) Dual class prototypes: Inner and hyper-dimensional prototypes are exploited to utilize the pre-trained information and obtain robust quasiorthogonal representations rather than example buffers for both privacy preservation and memory reduction. 2) Selfaugment and refinement: Instead of updating the whole network, we jointly optimize the extra projection module with the self-augment inner prototypes from base and novel classes, gradually refining the hyper-dimensional prototypes to obtain accurate decision boundaries for learned classes. Extensive experiments demonstrate the effectiveness and superiority of the proposed DSR in O 2 CL 1 .

show abstract