Multi-Agent Deep Reinforcement Learning with Human Strategies

Abstract

Deep learning has enabled traditional reinforcement learning methods to deal with highdimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increase the exploration capacity of multiple deep reinforcement learning agents. We also report the development of our own multiagent environment called Multiple Tank Defence to simulate the proposed approach. The results show the significant performance improvement of multiple agents that have learned cooperatively with human strategies. This implies that there is a critical need for human intellect teamed with machines to solve complex problems. In addition, the success of this simulation indicates that our multi-agent environment can be used as a testbed platform to develop and validate other multi-agent control algorithms.

Deep Reinforcement Learning for Multiagent Systems: A Review of Challenges, Solutions, and Applications

“…Establishing communication channels among agents during learning is an important step in designing and constructing MADRL algorithms. Nguyen et al [82] characterized the communication channel via human knowledge represented by images and allow deep RL agents to communicate using these shared images. The asynchronous advantage actor-critic (A3C) algorithm [74] is used to learn optimal policy for each agent, which can be extended to multiple heterogeneous agents.…”

“…These pose important research questions towards extensions of imitation learning and inverse RL to MADRL methods. In addition, for complicated tasks or behaviors which are difficult for humans to demonstrate, there is a need of alternative methods that allow human preferences to be integrated into deep RL [13,81,82].…”

Manipulating Soft Tissues by Deep Reinforcement Learning for Autonomous Robotic Surgery

“…As a result, it is possible to infer the trajectories into a dynamic environment based on the conditional distribution. Reinforcement learning (RL) has become a promising approach to modeling an autonomous agent [24]- [28]. RL has the abilities to mimic human learning behaviors to maximize the long-term reward.…”

Multi-Agent Reinforcement Learning Using Linear Fuzzy Model Applied to Cooperative Mobile Robots

“…The action policy is used to describe the agent's behavior, which specifies the way in which the agent chooses the action from a state. If the action policy, h = X → U, does not change over time it is considered stationary [20].…”