A continual learning survey: Defying forgetting in classification tasks

Abstract

Artificial neural networks thrive in solving the classification problem for a particular rigid task, where the network resembles a static entity of knowledge, acquired through generalized learning behaviour from a distinct training phase. However, endeavours to extend this knowledge without targeting the original task usually result in a catastrophic forgetting of this task. Continual learning shifts this paradigm towards a network that can continually accumulate knowledge over different tasks without the need for retraining from scratch, with methods in particular aiming to alleviate forgetting. We focus on task-incremental classification, where tasks arrive in a batch-like fashion, and are delineated by clear boundaries. Our main contributions concern 1) a taxonomy and extensive overview of the state-of-the-art, 2) a novel framework to continually determine stability-plasticity trade-off of the continual learner, 3) a comprehensive experimental comparison of 10 state-of-the-art continual learning methods and 4 baselines.We empirically scrutinize which method performs best, both on balanced Tiny Imagenet and a large-scale unbalanced iNaturalist datasets. We study the influence of model capacity, weight decay and dropout regularization, and the order in which the tasks are presented, and qualitatively compare methods in terms of required memory, computation time and storage. We make code publicly available upon acceptance of this paper.

Replay in Deep Learning: Current Approaches and Missing Biological Elements

“…Although humans and animals are able to continuously acquire new information over their lifetimes without catastrophically forgetting prior knowledge, artificial neural net-works lack these capabilities (Parisi et al, 2019;De Lange et al, 2019). Replay of previous experiences or memories in humans has been identified as the primary mechanism for overcoming forgetting and enabling continual knowledge acquisition (Walker and Stickgold, 2004).…”

“…Replay includes contents from both new and old memories networks Parisi et al, 2019;De Lange et al, 2019;, we provide the first comprehensive review that integrates and identifies the gaps between replay in these two fields. While it is beyond the scope of this paper to review everything known about the biology of replay, we highlight the salient differences between known biology and today's machine learning systems to help biologists test hypotheses and help machine learning researchers improve algorithms.…”

“…Three main types of methods for mitigating forgetting have been proposed (Parisi et al, 2019;De Lange et al, 2019): 1) regularization schemes for constraining weight updates with gradient descent (Kirkpatrick et al, 2017;Aljundi et al, 2018;Zenke et al, 2017;Chaudhry et al, 2018;Ritter et al, 2018;Serra et al, 2018;Dhar et al, 2019;Chaudhry et al, 2019;Lopez-Paz and Ranzato, 2017), 2) network expansion techniques for adding new parameters to a network to learn new information (Rusu et al, 2016;Yoon et al, 2018;Ostapenko et al, 2019;Hou et al, 2018), and 3) replay mechanisms for storing a representation of previous data to mix with new data when updating the network. Replay (or rehearsal) mechanisms have been shown to be the most effective of these approaches and are inspired by how the mammalian brain learns new information over time.…”

Generative Kernel Continual learning

“…Following Lange et al (2019), we divide continual learning methodologies into three different categories. We have regularisation-based methods (Kirkpatrick et al, 2017;Aljundi et al, 2018;Zenke et al, 2017;Kolouri et al, 2019) that regularize the neural network parameters to not change drastically from those learned on previous tasks.…”

Model architecture can transform catastrophic forgetting into positive transfer

“…In other words, when the system tried to optimize the loss function corresponding to the second task, it moved to a region of parameter space where it was not able to solve the first task. Inspired by human and animal learning [10][11][12][13][14][15][16] or even by the behavior of physical systems [17][18][19][20][21][22][23][24], modern approaches to this problem try to constrain the change of the parameters during training of the second task to avoid forgetting the first one [25][26][27][28][29][30][31][32][33][34][35].…”