CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions

Abstract

In the last few years, we have witnessed a renewed and fast-growing interest in continual learning with deep neural networks with the shared objective of making current AI systems more adaptive, efficient and autonomous. However, despite the significant and undoubted progress of the field in addressing the issue of catastrophic forgetting, benchmarking different continual learning approaches is a difficult task by itself. In fact, given the proliferation of different settings, training and evaluation protocols, metrics and nomenclature, it is often tricky to properly characterize a continual learning algorithm, relate it to other solutions and gauge its real-world applicability. The first Continual Learning in Computer Vision challenge held at CVPR in 2020 has been one of the first opportunities to evaluate different continual learning algorithms on a common hardware with a large set of shared evaluation metrics and 3 different settings based on the realistic CORe50 video benchmark. In this paper, we report the main results of the competition, which counted more than 79 teams registered, 11 finalists and 2300$ in prizes. We also summarize the winning approaches, current challenges and future research directions.

Recent Advances of Continual Learning in Computer Vision: An Overview

“…In recent years, an increasing number of continual learning methods have been proposed in various subareas of computer vision, as shown in Figure 1. Additionally, several competitions [9,10] related to continual learning in computer vision have been held in both 2020 and 2021. Hence, in this paper, we present an overview of the recent advances of continual learning in computer vision.…”

A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays

“…Concerning these latter scenarios, memory-based approaches, which preserve samples from previous tasks for replaying, perform better than regularization techniques, which simply address catastrophic forgetting by imposing constraints on the network parameter update at low memory cost [13]- [15]. This finding was confirmed during the recent CL competition at CVPR2020 [16], where the best entry leveraged on rehearsal based strategies.…”

“…Among these groups, rehearsal CL strategies have emerged as the most effective to deal with catastrophic forgetting, at the cost of an additional replay memory [1], [27], [28]. In the recent CL challenge at CVPR2020 on the Core50 image dataset, ∼90% of the competitors used rehearsal strategies [16]. The best entry of the more challenging New Instances and Classes track (the same scenario considered in our work) [17], which is evaluated in terms of test accuracy but also memory and computation requirements, scores 91% by replaying image data.…”

“…Conversely, the Latent Replay-based approach [1] relies on a fixed, and relatively small, amount of compressed latent activations as replay data; it scores 71% if retraining only the last layer, which presents a peak of 52× lower (compressed) data points than the winning solution. Additionally, the Jodelet entry -also employing LR-based CL -achieves 83% thanks to 3× more replays and a more accurate pre-trained model (ResNet50) [16]. In our work, we focus on [1] because of the tunable accuracy-memory setting.…”

Online Continual Learning in Image Classification: An Empirical Survey

“…The size of the mini-batch retrieved from the memory buffer is also set to 10, irrespective of the size of the memory buffer as in [34]. Note that with techniques such as transfer learning (e.g., using a pre-trained model from ImageNet), data augmentation and deeper network architectures, it is possible to achieve much higher performance in this setting [109]. However, since those techniques are orthogonal to our investigation and deviate from the simpler experimental settings of other papers we cite and compare, we do not use them in our experiments.…”