Information-theoretic Online Memory Selection for Continual Learning

Abstract

A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theoretic perspective. To gather the most information, we propose the surprise and the learnability criteria to pick informative points and to avoid outliers. We present a Bayesian model to compute the criteria efficiently by exploiting rank-one matrix structures. We demonstrate that these criteria encourage selecting informative points in a greedy algorithm for online memory selection. Furthermore, by identifying the importance of the timing to update the memory, we introduce a stochastic informationtheoretic reservoir sampler (InfoRS), which conducts sampling among selective points with high information. Compared to reservoir sampling, InfoRS demonstrates improved robustness against data imbalance. Finally, empirical performances over continual learning benchmarks manifest its efficiency and efficacy.

Continual Variational Autoencoder via Continual Generative Knowledge Distillation

“…Continual learning : Most existing approaches to CL require access to task information during training. There are three categories of CL approaches : Regularisation-based methods (Hinton, Vinyals, and Dean 2014;Kirkpatrick et al 2017;Kurle et al 2020;Li and Hoiem 2017;Nguyen et al 2018;Polikar et al 2001;Ren et al 2017;Ritter, Botev, and Barber 2018), dynamic architectures (Fernando et al 2017;Golkar, Kagan, and Cho 2019;Hung et al 2019;Rusu et al 2016;Wen, Tran, and Ba 2020;Ye and Bors 2023, 2021a, 2022d and memory-based approaches (Achille et al 2018;Ramapuram, Gregorova, and Kalousis 2017;Rao et al 2019;Shin et al 2017;Sun et al 2022;Bors 2020a,b, 2022e;Zhai et al 2019;Ye and Bors 2021b;Yoon et al 2022). Because of requiring the task information (Aljundi et al 2019a) these approaches cannot be applied directly to TFCL.…”