Transferring Instances for Model-Based Reinforcement Learning

Taylor, Matthew E.; Jong, Nicholas K.; Stone, Peter

doi:10.1007/978-3-540-87481-2_32

Cited by 101 publications

(85 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The type of knowledge that can be transferred between tasks varies among different TL methods, including value functions [8], entire policies [9], actions (policy advice) [10], or a set of samples from a source task that can be used by a model-based RL algorithm in a target task [11].…”

Section: Transfer Learning and Advising Under A Budgetmentioning

confidence: 99%

Learning to Teach Reinforcement Learning Agents

Fachantidis

Taylor

Vlahavas

2017

MAKE

View full text Add to dashboard Cite

Abstract:In this article, we study the transfer learning model of action advice under a budget. We focus on reinforcement learning teachers providing action advice to heterogeneous students playing the game of Pac-Man under a limited advice budget. First, we examine several critical factors affecting advice quality in this setting, such as the average performance of the teacher, its variance and the importance of reward discounting in advising. The experiments show that the best performers are not always the best teachers and reveal the non-trivial importance of the coefficient of variation (CV) as a statistic for choosing policies that generate advice. The CV statistic relates variance to the corresponding mean. Second, the article studies policy learning for distributing advice under a budget. Whereas most methods in the relevant literature rely on heuristics for advice distribution, we formulate the problem as a learning one and propose a novel reinforcement learning algorithm capable of learning when to advise or not. The proposed algorithm is able to advise even when it does not have knowledge of the student's intended action and needs significantly less training time compared to previous learning approaches. Finally, in this article, we argue that learning to advise under a budget is an instance of a more generic learning problem: Constrained Exploitation Reinforcement Learning.

show abstract

Section: Transfer Learning and Advising Under A Budgetmentioning

confidence: 99%

Learning to Teach Reinforcement Learning Agents

Fachantidis

Taylor

Vlahavas

2017

MAKE

View full text Add to dashboard Cite

show abstract

“…In order to cluster tasks into different classes, Lazaric and Ghavamzadeh (2010) place Fig. 6 Transfer from 2D to 3D mountain car (Taylor et al, 2008a). …”

Section: Parameter Transfermentioning

confidence: 99%

Transfer in Reinforcement Learning: A Framework and a Survey

Lazaric

2012

Adaptation, Learning, and Optimization

219

146

View full text Add to dashboard Cite

Transfer in reinforcement learning is a novel research area that focuses on the development of methods to transfer knowledge from a set of source tasks to a target task. Whenever the tasks are similar, the transferred knowledge can be used by a learning algorithm to solve the target task and significantly improve its performance (e.g., by reducing the number of samples needed to achieve a nearly optimal performance). In this chapter we provide a formalization of the general transfer problem, we identify the main settings which have been investigated so far, and we review the most important approaches to transfer in reinforcement learning.

show abstract

“…The level of knowledge that can be transferred across tasks can be low, such as tuples of the form s, a, r, s ′ [6,10], value-functions [12] or policies [2]. Higher level knowledge may include rules [7,13], action subsets or shaping rewards [5].…”

Section: Transfer Learning In Rlmentioning

confidence: 99%

Transfer Learning in Multi-Agent Reinforcement Learning Domains

Boutsioukis

Partalas

Vlahavas

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. Transfer learning refers to the process of reusing knowledge from past tasks in order to speed up the learning procedure in new tasks. In reinforcement learning, where agents often require a considerable amount of training, transfer learning comprises a suitable solution for speeding up learning. Transfer learning methods have primarily been applied in single-agent reinforcement learning algorithms, while no prior work has addressed this issue in the case of multi-agent learning. This work proposes a novel method for transfer learning in multi-agent reinforcement learning domains. We test the proposed approach in a multiagent domain under various setups. The results demonstrate that the method helps to reduce the learning time and increase the asymptotic performance.

show abstract

Transferring Instances for Model-Based Reinforcement Learning

Cited by 101 publications

References 15 publications

Learning to Teach Reinforcement Learning Agents

Learning to Teach Reinforcement Learning Agents

Transfer in Reinforcement Learning: A Framework and a Survey

Transfer Learning in Multi-Agent Reinforcement Learning Domains

Contact Info

Product

Resources

About