Multiagent Transfer Learning via Assignment-Based Decomposition

Proper, Scott; Tadepalli, Prasad

doi:10.1109/icmla.2009.59

Cited by 5 publications

(3 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The required description is intuitive to give and could be easily adaptable to cope with the multiagent case (each type of agent could be a class as in Multiagent OO-MDPs). Proper and Tadepalli (2009) also make use of a relational representation to transfer knowledge between tasks. In their proposal (specialized for MAS), a group of agents is assigned to solve collaboratively a subtask, while ignoring all agents assigned to different subtasks.…”

Section: Relational Descriptionsmentioning

confidence: 99%

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Silva¹,

Costa²

2019

jair

211

106

View full text Add to dashboard Cite

Multiagent Reinforcement Learning (RL) solves complex tasks that require coordination with other agents through autonomous exploration of the environment. However, learning a complex task from scratch is impractical due to the huge sample complexity of RL algorithms. For this reason, reusing knowledge that can come from previous experience or other agents is indispensable to scale up multiagent RL algorithms. This survey provides a unifying view of the literature on knowledge reuse in multiagent RL. We define a taxonomy of solutions for the general knowledge reuse problem, providing a comprehensive discussion of recent progress on knowledge reuse in Multiagent Systems (MAS) and of techniques for knowledge reuse across agents (that may be actuating in a shared environment or not). We aim at encouraging the community to work towards reusing all the knowledge sources available in a MAS. For that, we provide an in-depth discussion of current lines of research and open questions.

show abstract

Section: Relational Descriptionsmentioning

confidence: 99%

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Silva¹,

Costa²

2019

jair

211

106

View full text Add to dashboard Cite

show abstract

“…Moreover, this would also establish a linkage to learning techniques that are closely related to transfer learning such as imitation learning (for example, Price and Boutilier [2003]), learning from demonstration (for example, Breazeal et al [2006]) and multitask learning (for example, Ando and Zhang [2005]). Currently very little multiagent transfer learning work is available (for example, Ammar and Taylor [2011], Proper and Tadepalli [2009], Wilson et al [2008], Wilson, Fern, and Tadepalli [2010]).…”

Section: Extending the Scopementioning

confidence: 99%

Multiagent Learning: Basics, Challenges, and Prospects

2012

View full text Add to dashboard Cite

Multiagent systems (MAS) are widely accepted as an important method for solving problems of a distributed nature. A key to the success of MAS is efficient and effective multiagent learning (MAL). The past twenty-five years have seen a great interest and tremendous progress in the field of MAL. This article introduces and overviews this field by presenting its fundamentals, sketching its historical development and describing some key algorithms for MAL. Moreover, main challenges that the field is facing today are indentified.

show abstract

“…While relational approaches have benefited MAS in previous work (CROONEN-BORGHS et al, 2005;PROPER;TADEPALLI, 2009), OO-MDP has not been applied in multiagent RL so far. To the best of our knowledge, up to now the only effort toward extending OO-MDP to MAS appears in BURLAP (MACGLASHAN, 2015), a library for the development of planning and learning algorithms based on OO-MDP.…”

Section: Object-oriented Representation In Masmentioning

confidence: 99%

Methods and algorithms for knowledge reuse in multiagent reinforcement learning.

Silva¹

View full text Add to dashboard Cite

Reinforcement Learning (RL) is a well-known technique to train autonomous agents through interactions with the environment. However, the learning process has a high sample-complexity to infer an effective policy, especially when multiple agents are simultaneously actuating in the environment. We here propose to take advantage of previous knowledge, so as to accelerate learning in multiagent RL problems. Agents may reuse knowledge gathered from previously solved tasks, and they may also receive guidance from more experienced friendly agents to learn faster. However, specifying a framework to integrate knowledge reuse into the learning process requires answering challenging research questions, such as: How to abstract task solutions to reuse them later in similar yet different tasks? How to define when advice should be given? How to select the previous task most similar to the new one and map correspondences? and How to defined if received advice is trustworthy? Although many methods exist to reuse knowledge from a specific knowledge source, the literature is composed of methods very specialized in their own scenario that are not compatible. We propose in this thesis to reuse knowledge both from previously solved tasks and from communication with other agents. In order to accomplish our goal, we propose several flexible methods to enable each of those two types of knowledge reuse. Our proposed methods include: Ad Hoc Advising, an inter-agent advising framework, where agents can share knowledge among themselves through action suggestions; and an extension of the object-oriented representation to multiagent RL and methods to leverage it for reusing knowledge. Combined, our methods provide ways to reuse knowledge from both previously solved tasks and other agents with state-of-the-art performance. Our contributions are first steps towards more flexible and broadly applicable multiagent transfer learning methods, where agents will be able to consistently combine reused knowledge from multiple sources, including solved tasks and other learning agents.

show abstract

Multiagent Transfer Learning via Assignment-Based Decomposition

Cited by 5 publications

References 4 publications

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Multiagent Learning: Basics, Challenges, and Prospects

Methods and algorithms for knowledge reuse in multiagent reinforcement learning.

Contact Info

Product

Resources

About