A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Silva, Felipe Leno da; Costa, Anna Helena Reali

doi:10.1613/jair.1.11396

Cited by 211 publications

(127 citation statements)

References 112 publications

Supporting

Mentioning

119

Contrasting

Unclassified

Order By: Relevance

“…Since then, the number of published MAL works continues to steadily rise, which led to different surveys on the area, ranging from analyzing the basics of MAL and their challenges [3,4,5], to addressing specific subareas: game theory and MAL [2,6], cooperative scenarios [7,8], and evolutionary dynamics of MAL [9]. In just the last couple of years, three surveys related to MAL have been published: learning in non-stationary environments [10], agents modeling agents [11], and transfer learning in multiagent RL [12].…”

Section: Introductionmentioning

confidence: 99%

“…Despite this complexity, top AI conferences like AAAI, ICML, ICLR, IJCAI and NeurIPS, and specialized conferences such as AAMAS, have published works reporting successes in MDRL. In light of these works, we believe it is pertinent to first, have an overview of the recent MDRL works, and second, understand how these recent works relate to the existing literature.This article contributes to the state of the art with a brief survey of the current works in MDRL in an effort to complement existing surveys on multiagent learning [36,10], cooperative learning [7,8], agents modeling agents [11], knowledge reuse in multiagent RL [12], and (singleagent) deep reinforcement learning [23,37].First, we provide a short review of key algorithms in RL such as Q-learning and REINFORCE (see Section 2.1). Second, we review DRL highlighting the challenges in this setting and reviewing recent works (see Section 2.2).…”

mentioning

confidence: 99%

See 1 more Smart Citation

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

437

240

View full text Add to dashboard Cite

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.$ Earlier versions of this work had the title: "Is multiagent deep reinforcement learning the answer or the question? A brief survey" arXiv:1810.05587v3 [cs.MA] 30 Aug 2019 Go [14,15], poker [16,17], and games of two competing teams, e.g., DOTA 2 [18] and StarCraft II [19].While different techniques and algorithms were used in the above scenarios, in general, they are all a combination of techniques from two main areas: reinforcement learning (RL) [20] and deep learning [21,22].RL is an area of machine learning where an agent learns by interacting (i.e., taking actions) within a dynamic environment. However, one of the main challenges to RL, and traditional machine learning in general, is the need for manually designing quality features on which to learn. Deep learning enables efficient representation learning, thus allowing the automatic discovery of features [21,22]. In recent years, deep learning has had successes in different areas such as computer vision and natural language processing [21,22]. One of the key aspects of deep learning is the use of neural networks (NNs) that can find compact representations in high-dimensional data [23].In deep reinforcement learning (DRL) [23,24] deep neural networks are trained to approximate the optimal policy and/or the value function. In this way the deep NN, serving as function approximator, enables powerful generalization. One of the key advantages of DRL is that it enables RL to scale to problems with high-dimensional state and action spaces. However, most existing successful DRL applications so far have been on visual domains (e.g., Atari games), and there is still a lot of work to be done for more realistic applications [25,26] with complex dynamics, which are not necessarily vision-based.DRL h...

show abstract

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

A survey and critique of multiagent deep reinforcement learning

Hernández-Leal¹,

Kartal²,

Taylor³

2019

Auton Agent Multi-Agent Syst

437

240

View full text Add to dashboard Cite

show abstract

“…The effectiveness of learning in CNN models can be improved even further. There are many important factors to consider, such as improving model weight initialization by transfer learning or using data augmentation and dropout as methods of regularization to combat overfitting during model training [ 16 , 17 , 18 ]. In training CNN models, a large dataset is needed for the model to learn the patterns of features that are complex in detail so that the CNN model can classify those features, achieving an appropriate classification performance [ 19 , 20 ].…”

Section: Introductionmentioning

confidence: 99%

Analyzing Malaria Disease Using Effective Deep Learning Approach

2020

View full text Add to dashboard Cite

Medical tools used to bolster decision-making by medical specialists who offer malaria treatment include image processing equipment and a computer-aided diagnostic system. Malaria images can be employed to identify and detect malaria using these methods, in order to monitor the symptoms of malaria patients, although there may be atypical cases that need more time for an assessment. This research used 7000 images of Xception, Inception-V3, ResNet-50, NasNetMobile, VGG-16 and AlexNet models for verification and analysis. These are prevalent models that classify the image precision and use a rotational method to improve the performance of validation and the training dataset with convolutional neural network models. Xception, using the state of the art activation function (Mish) and optimizer (Nadam), improved the effectiveness, as found by the outcomes of the convolutional neural model evaluation of these models for classifying the malaria disease from thin blood smear images. In terms of the performance, recall, accuracy, precision, and F1 measure, a combined score of 99.28% was achieved. Consequently, 10% of all non-dataset training and testing images were evaluated utilizing this pattern. Notable aspects for the improvement of a computer-aided diagnostic to produce an optimum malaria detection approach have been found, supported by a 98.86% accuracy level.

show abstract

“…The automatic generation of curricula [Da Silva and Costa, 2019] has been divided into two sub-problems: task generation [Narvekar et al, 2016;Da Silva and Costa, 2018], that is the problem of creating a set of tasks such that transferring from them is most likely beneficial for the final task; and task sequencing [Svetlik et al, 2017;Narvekar et al, 2017;Da Silva and Costa, 2018;Foglino et al, 2019], whereby previously generated tasks are optimally selected and ordered. Current methods for task sequencing attempt to determine the optimal order of tasks either with [Narvekar et al, 2017;Baranes and Oudeyer, 2013] or without [Svetlik et al, 2017;Da Silva and Costa, 2018] executing the tasks.…”

Section: Related Workmentioning

confidence: 99%

Curriculum Learning for Cumulative Return Maximization

Foglino

Christakou

Gutierrez

et al. 2019

Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence

View full text Add to dashboard Cite

Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape exploration through transfer. We propose a task sequencing algorithm maximizing the cumulative return, that is, the return obtained by the agent across all the learning episodes. By maximizing the cumulative return, the agent not only aims at achieving high rewards as fast as possible, but also at doing so while limiting suboptimal actions. We experimentally compare our task sequencing algorithm to several popular metaheuristic algorithms for combinatorial optimization, and show that it achieves significantly better performance on the problem of cumulative return maximization. Furthermore, we validate our algorithm on a critical task, optimizing a home controller for a micro energy grid.

show abstract

A Survey on Transfer Learning for Multiagent Reinforcement Learning Systems

Cited by 211 publications

References 112 publications

A survey and critique of multiagent deep reinforcement learning

A survey and critique of multiagent deep reinforcement learning

Analyzing Malaria Disease Using Effective Deep Learning Approach

Curriculum Learning for Cumulative Return Maximization

Contact Info

Product

Resources

About