Teacher-Student Curriculum Learning

Matiisen, Tambet; Oliver, Avital; Cohen, Taco; John, Sabu

doi:10.48550/arxiv.1707.00183

Cited by 31 publications

(42 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Task weights are controlled by the ratio between a recent training loss and the loss at a previous time step, so that tasks that progress faster will be downweighted, while straggling ones will be upweighted. This approach contrasts with the curriculum learning framework proposed by Matiisen et al [14], where tasks with faster progress are preferred. Loss progress, and well as a few other signals, were also employed by Graves et al [6], which formulated curriculum learning as a multi-armed bandit problem.…”

Section: Discussion and Other Related Workmentioning

confidence: 96%

Adaptive Scheduling for Multi-Task Learning

Jean,

Firat,

Johnson

2019

Preprint

View full text Add to dashboard Cite

To train neural machine translation models simultaneously on multiple tasks (languages), it is common to sample each task uniformly or in proportion to dataset sizes. As these methods offer little control over performance trade-offs, we explore different task scheduling approaches. We first consider existing non-adaptive techniques, then move on to adaptive schedules that over-sample tasks with poorer results compared to their respective baseline. As explicit schedules can be inefficient, especially if one task is highly over-sampled, we also consider implicit schedules, learning to scale learning rates or gradients of individual tasks instead. These techniques allow training multilingual models that perform better for lowresource language pairs (tasks with small amount of data), while minimizing negative effects on high-resource tasks.

show abstract

Section: Discussion and Other Related Workmentioning

confidence: 96%

Adaptive Scheduling for Multi-Task Learning

Jean,

Firat,

Johnson

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The second step involves the exploration decision. We may for example select a new goal based on learning progress (Baranes and Oudeyer, 2013;Matiisen et al, 2017), selecting the goal which has shown the largest recent change in our ability to reach it. Otherwise, we can also train a generative model on the state that were of intermediate difficulty to reach, and sample a next goal from this model (Florensa et al, 2018).…”

Section: Explorationmentioning

confidence: 99%

A Unifying Framework for Reinforcement Learning and Planning

Moerland¹,

Broekens²,

Plaat³

et al. 2020

Preprint

View full text Add to dashboard Cite

Sequential decision making, commonly formalized as Markov Decision Process optimization, is a key challenge in artificial intelligence. Two successful approaches to MDP optimization are planning and reinforcement learning. Both research fields largely have their own research communities. However, if both research fields solve the same problem, then we should be able to disentangle the common factors in their solution approaches. Therefore, this paper presents a unifying framework for reinforcement learning and planning (FRAP), which identifies the underlying dimensions on which any planning or learning algorithm has to decide. At the end of the paper, we compare -in a single table -a variety of well-known planning, model-free and model-based RL algorithms along the dimensions of our framework, illustrating the validity of the framework. Altogether, FRAP provides deeper insight into the algorithmic space of planning and reinforcement learning, and also suggests new approaches to integration of both fields.

show abstract

“…Automatic curriculum generation systems often assume that some tasks cannot be tackled until other easier tasks have been solved first. Curriculum has been most explored in the supervised multi-task learning case where the agent can switch between tasks at any moment (Graves et al, 2017;Matiisen et al, 2017). Although we could use some ideas from curriculum learning in our bandit-like task, these approaches would not be compatible with more general sequential decision making tasks with delayed outcomes (e.g., a robot), which represents our ultimate goal.…”

Section: Intrinsic Rewards For Multi-prediction Learningmentioning

confidence: 99%

Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study

Linke,

Ady,

White

et al. 2019

Preprint

View full text Add to dashboard Cite

Learning about many things can provide numerous benefits to a reinforcement learning system. For example, learning many auxiliary value functions, in addition to optimizing the environmental reward, appears to improve both exploration and representation learning. The question we tackle in this paper is how to sculpt the stream of experience-how to adapt the system's behaviour-to optimize the learning of a collection of value functions. A simple answer is to compute an intrinsic reward based on the statistics of each auxiliary learner, and use reinforcement learning to maximize that intrinsic reward. Unfortunately, implementing this simple idea has proven difficult, and thus has been the focus of decades of study. It remains unclear which of the many possible measures of learning would work well in a parallel learning setting where environmental reward is extremely sparse or absent. In this paper, we investigate and compare different intrinsic reward mechanisms in a new bandit-like parallel-learning testbed. We discuss the interaction between reward and prediction learners and highlight the importance of introspective prediction learners: those that increase their rate of learning when progress is possible, and decrease when it is not. We provide a comprehensive empirical comparison of 15 different rewards, including well-known ideas from reinforcement learning and active learning. Our results highlight a simple but seemingly powerful principle: intrinsic rewards based on the amount of learning can generate useful behaviour, if each individual learner is introspective.

show abstract

Teacher-Student Curriculum Learning

Cited by 31 publications

References 0 publications

Adaptive Scheduling for Multi-Task Learning

Adaptive Scheduling for Multi-Task Learning

A Unifying Framework for Reinforcement Learning and Planning

Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study

Contact Info

Product

Resources

About