La-MAML: Look-ahead Meta Learning for Continual Learning

Abstract

The continual learning problem involves training models with limited capacity to perform well on a set of an unknown number of sequentially arriving tasks. While meta-learning shows great potential for reducing interference between old and new tasks, the current training procedures tend to be either slow or offline, and sensitive to many hyper-parameters. In this work, we propose Look-ahead MAML (La-MAML), a fast optimisation-based meta-learning algorithm for onlinecontinual learning, aided by a small episodic memory. Our proposed modulation of per-parameter learning rates in our meta-learning update allows us to draw connections to prior work on hypergradients and meta-descent. This provides a more flexible and efficient way to mitigate catastrophic forgetting compared to conventional prior-based methods. La-MAML achieves performance superior to other replay-based, prior-based and meta-learning based approaches for continual learning on real-world visual classification benchmarks. * equal contribution Preprint. Under review.

Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning

“…Intuitively, a basis will be regarded as important for preserving old knowledge if the gradients induced by new tasks and old ones are aligned in the opposite direction on that basis. As demonstrated through extensive experiments, our proposed method can consistently outperform the state-of-the-art methods [17,30,24,31,5,13,34] by a notable margin across a range of widely used benchmark datasets.…”

“…A few other works have utilized gradient information to protect previous knowledge. [31,13] adopt optimization-based meta-learning to enforce gradient alignment between samples from different tasks. GPM [34] minimizes interference between sequential tasks by ensuring that gradient updates only occur in directions orthogonal to the input of previous tasks.…”

“…We first study the stability of the network by plotting changes of the weight loss landscape for the first task after new task learning. In particular, We use the previously proposed ER [5], La-MAML [13], and GPM [34] to train a MLP network with two hidden layers on the Permuted MNIST (PMNIST) [20] dataset that contains 10 tasks. We also retrain the network on the entire dataset contain all passed tasks as an Oracle network.…”

“…As presented in the neurocognitive works [41,15], the reactivation of neuronal activity patterns, representing old memories, plays an important role in the continual learning of humans [39]. Thus, forgetness can be effectively mitigated by training a single network for new tasks by considering diverse information stored in the memory, including the original training samples of old tasks [31,5,13], the gradients induced from old tasks [9] and the feature subspace representing old tasks [34]. However, their continual learning performance is still limited because DNNs can easily overfit the limited information stored in the small-size memory.…”

Reviewing continual learning from the perspective of human-level intelligence

“…In addition to OML, MAML [66] and its variant FOMAML [66] have been utilized. For example, Gupta et al [68] optimize the OML objective in an online way through a multistep MAML procedure. The authors indicate that the gradient alignment among old tasks does not degrade while a new task is learned in OML; therefore, it is necessary to avoid repeated optimization of the inter-task alignment between old tasks to enable acceleration.…”