Continual learning via inter-task synaptic mapping

Mao, Fubing; Weng, Weiwei; Pratama, Mahardhika; Yapp, Edward Kien Yee

doi:10.1016/j.knosys.2021.106947

Cited by 17 publications

(1 citation statement)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Numerous CL algorithms have been developed in the literature (Srivastava et al, 2013;Farquhar & Gal, 2018;Kim et al, 2018;Morgado & Vasconcelos, 2019;Benavides-Prado et al, 2020;Parisi & Lomonaco, 2020;Ahn et al, 2021;Ayub & Wagner, 2021;Cha et al, 2021a;Derakhshani et al, 2021;Ehret et al, 2021;Hurtado et al, 2021;Kapoor et al, 2021;Mao et al, 2021;Tang & Matteson, 2021;Yoon et al, 2021;Benavides-Prado & Riddle, 2022;Madaan et al, 2022;Ramesh & Chaudhari, 2022;Romero et al, 2022a;Skantze & Willemsen, 2022;Wang et al, 2022b;Gaya et al, 2023;Mundt et al, 2023). On a high level, there are three principal approaches to continual learning (CL): memory-based, regularization-based and architecture-based (Pan et al, 2020;Parisi & Lomonaco, 2020;Krishnan & Balaprakash, 2021;Mehta et al, 2021;.…”

Section: Related Workmentioning

confidence: 99%

Similarity-Based Adaptation for Task-Aware and Task-Free Continual Learning

Adel

2024

jair

View full text Add to dashboard Cite

Continual learning (CL) is a paradigm which addresses the issue of how to learn from sequentially arriving tasks. The goal of this paper is to introduce a CL framework which can both learn from a global multi-task architecture and locally adapt this learning to the task at hand. In addition to the global knowledge, we conjecture that it is also beneficial to further focus on the most relevant pieces of previous knowledge. Using a prototypical network as a proxy, the proposed framework bases its adaptation on the similarity between the current data stream and the previously encountered data. We develop two algorithms, one for the standard task-aware CL and another for the more challenging task-free setting where boundaries between tasks are unknown. We correspondingly derive a generalization upper bound on the error of an upcoming task. Experiments demonstrate that the introduced algorithms lead to improved performance on several CL benchmarks.

show abstract