Active Task Selection for Lifelong Machine Learning

Ruvolo, Paul; Eaton, Eric

doi:10.1609/aaai.v27i1.8684

Cited by 41 publications

(52 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The order in which tasks are encountered does impact learning, and several groups have investigated the effects of ordering (Bengio et al 2009;Ruvolo and Eaton 2013a;Narvekar et al 2016). For our experiments, we selected a random task ordering that demonstrates forgetting and held it fixed for all experiments.…”

Section: Methodsmentioning

confidence: 99%

Selective Experience Replay for Lifelong Learning

Isele

Cosgun

2018

AAAI

213

View full text Add to dashboard Cite

Deep reinforcement learning has emerged as a powerful tool for a variety of learning tasks, however deep nets typically exhibit forgetting when learning multiple tasks in sequence. To mitigate forgetting, we propose an experience replay process that augments the standard FIFO buffer and selectively stores experiences in a long-term memory. We explore four strategies for selecting which experiences will be stored: favoring surprise, favoring reward, matching the global training distribution, and maximizing coverage of the state space. We show that distribution matching successfully prevents catastrophic forgetting, and is consistently the best approach on all domains tested. While distribution matching has better and more consistent performance, we identify one case in which coverage maximization is beneficial---when tasks that receive less trained are more important. Overall, our results show that selective experience replay, when suitable selection algorithms are employed, can prevent catastrophic forgetting.

show abstract

Section: Methodsmentioning

confidence: 99%

Selective Experience Replay for Lifelong Learning

Isele

Cosgun

2018

AAAI

213

View full text Add to dashboard Cite

show abstract

“…However, ELLA [32] was introduced as a general lifelong learning algorithm that operates within a multi-task learning framework, allowing the model to learn multiple basic learning models continuously. This led to the development of probability-based [33] and non-parametric Bayesian methods [34], which share information between tasks through linear combinations of basis vectors to enhance model performance. However, these methods have limitations in terms of the types of learning tasks they can handle.…”

Section: Related Workmentioning

confidence: 99%

Meta-Learning Based Tasks Similarity Representation for Cross Domain Lifelong Learning

Shen¹,

Chen

Ren

2023

IEEE Access

View full text Add to dashboard Cite

Deep neural networks perform better in most specific single tasks than humans, but it is hard to handle a sequence of new tasks from different domains. The deep learning-based models always need to remember the parameters of the learned tasks to perform well in the new tasks and forfeit the ability to generalize from previous data, which is inconsistent with human learning. We propose a novel lifelong learning framework that can guide the model to learn new knowledge without forgetting the old knowledge through learning the similarity representation based on meta-learning. Specifically, we employ a cross-domain triplets network (CDTN) by minimizing the maximum mean discrepancy (MMD) between the current task and the knowledge base to learn the domain invariant similarity representation between tasks in different domains. Furthermore, we add a self-attention module to enhance the extraction of similarity features. Secondly, a spatial attention network (SAN) which can assign different weights according to the learned similarity representation of tasks, is proposed in addition. The experimental results show that our method effectively reduces catastrophic forgetting compared with the state-of-the-art methods when learning many tasks. Moreover, we show that the proposed method can hardly forget the old knowledge while continuously enhancing the performance of the old tasks, which is more in line with the human way of learning.

show abstract

“…However, none of these studies consider the problem of meta-learning with a fixed budget. A few studies have looked into actively choosing the next task in a sequence of tasks (Ruvolo and Eaton 2013), (Pentina, Sharmanska, and Lampert 2015), (Pentina and Lampert 2017), (Sun, Cong, and Xu 2018), but they do not look at how to distribute data across tasks.…”

Section: Related Workmentioning

confidence: 99%

How to Distribute Data across Tasks for Meta-Learning?

Cioba

Bromberg

Wang

et al. 2022

AAAI

View full text Add to dashboard Cite

Meta-learning models transfer the knowledge acquired from previous tasks to quickly learn new ones. They are trained on benchmarks with a fixed number of data points per task. This number is usually arbitrary and it is unknown how it affects performance at testing. Since labelling of data is expensive, finding the optimal allocation of labels across training tasks may reduce costs. Given a fixed budget of labels, should we use a small number of highly labelled tasks, or many tasks with few labels each? Should we allocate more labels to some tasks and less to others? We show that: 1) If tasks are homogeneous, there is a uniform optimal allocation, whereby all tasks get the same amount of data; 2) At fixed budget, there is a trade-off between number of tasks and number of data points per task, with a unique solution for the optimum; 3) When trained separately, harder task should get more data, at the cost of a smaller number of tasks; 4) When training on a mixture of easy and hard tasks, more data should be allocated to easy tasks. Interestingly, Neuroscience experiments have shown that human visual skills also transfer better from easy tasks. We prove these results mathematically on mixed linear regression, and we show empirically that the same results hold for few-shot image classification on CIFAR-FS and mini-ImageNet. Our results provide guidance for allocating labels across tasks when collecting data for meta-learning.

show abstract

Active Task Selection for Lifelong Machine Learning

Cited by 41 publications

References 11 publications

Selective Experience Replay for Lifelong Learning

Selective Experience Replay for Lifelong Learning

Meta-Learning Based Tasks Similarity Representation for Cross Domain Lifelong Learning

How to Distribute Data across Tasks for Meta-Learning?

Contact Info

Product

Resources

About