What is Happening Inside a Continual Learning Model? - A Representation-Based Evaluation of Representational Forgetting -

Feature Forgetting in Continual Representation Learning

“…Similar to commonly observed forgetting on the output level, forgetting on the feature level can also happen in continual learning, although being more elusive for analysis [10]. Changes in the representation does not necessarily mean forgetting.…”

“…Besides being a natural extension of a common practice in representation evaluation, this protocol has the following advantages: 1) Consistency: the choice of evaluation tasks is unambiguous and consistent, independent of the model and the algorithm, unlike in general representation evaluation and in [10]. This makes the metric P rep good for comparing methods, even against non-continual learning algorithms.…”

“…However, even with a catastrophic drop in performance on past tasks, the learned representation seems still largely intact [8,9]. The problem of evaluating representation forgetting is raised in [10], which refers specifically to the degradation of learned features rather than the degradation of final prediction performance. Because the learned features are crucial for performance on both past and future tasks, it is important to make sure the features are learned well and forgetting is minimized.…”

“…Previous work [10,29] has used a re-training strategy to evaluate representations in continual learning on past tasks to examine the performance of lower layer representations. However, re-evaluating on past tasks risks confouding representation learning with catastrophic forgetting.…”