Pommerman: A Multi-Agent Playground

Resnick, Cinjon; Eldridge, Wes; Ha, David; Britz, Denny; Foerster, Jakob; Togelius, Julian; Cho, Kyunghyun; Bruna, Joan

doi:10.48550/arxiv.1809.07124

Cited by 27 publications

(47 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While multi-agent reinforcement learning (MARL) is a well-established branch of Deep RL, most learning algorithms and environments proposed have targeted a relatively small number of agents 17,41 . It is common to see environments with less dozens of agents 1,30,52,67 , with 2-agent and 4-agent environments being particular popular for the study of competitive, self-play settings 2,23,36 . Collective intelligence observed in nature, however, rely on a much larger number of individuals than typically studied in MARL, involving population sizes from thousands to million.…”

Section: Multi-agent Learningmentioning

confidence: 99%

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Ha¹,

Tang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In the past decade, we have witnessed the rise of deep learning to dominate the field of artificial intelligence. Advances in artificial neural networks alongside corresponding advances in hardware accelerators with large memory capacity, together with the availability of large datasets enabled researchers and practitioners alike to train and deploy sophisticated neural network models that achieve state-of-the-art performance on tasks across several fields spanning computer vision, natural language processing, and reinforcement learning. However, as these neural networks become bigger, more complex, and more widely used, fundamental problems with current deep learning models become more apparent. State-of-the-art deep learning models are known to suffer from issues that range from poor robustness, inability to adapt to novel task settings, to requiring rigid and inflexible configuration assumptions. Ideas from collective intelligence, in particular concepts from complex systems such as self-organization, emergent behavior, swarm optimization, and cellular systems tend to produce solutions that are robust, adaptable, and have less rigid assumptions about the environment configuration. It is therefore natural to see these ideas incorporated into newer deep learning methods. In this review, we will provide a historical context of neural network research's involvement with complex systems, and highlight several active areas in modern deep learning research that incorporate the principles of collective intelligence to advance its current capabilities. To facilitate a bi-directional flow of ideas, we also discuss work that utilize modern deep learning models to help advance complex systems research. We hope this review can serve as a bridge between complex systems and deep learning communities to facilitate the cross pollination of ideas and foster new collaborations across disciplines.

show abstract

Section: Multi-agent Learningmentioning

confidence: 99%

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Ha¹,

Tang²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Pommerman [17] is a variant of the famous game Bomberman and is used as a benchmark for multi-agent learning. Typically, there are 4 agents that each can move and place bomb on an 11 × 11 board.…”

Section: Pommermanmentioning

confidence: 99%

“…Then we describe CSP-MARL and explain the design of our code implementation in Section 3. Finally, we discuss several experiments over StarCraft 2, ViZDoom [16] and Pommerman [17] in Section 4 to show the efficiency and effectiveness of TLeague.…”

Section: Introductionmentioning

confidence: 99%

TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

Sun,

Xiong,

Han

et al. 2020

Preprint

View full text Add to dashboard Cite

Competitive Self-Play (CSP) based Multi-Agent Reinforcement Learning (MARL) has shown phenomenal breakthroughs recently. Strong AIs are achieved for several benchmarks, including Dota 2, Glory of Kings, Quake III, StarCraft II, to name a few. Despite the success, the MARL training is extremely data thirsty, requiring typically billions of (if not trillions of) frames be seen from the environment during training in order for learning a high performance agent. This poses non-trivial difficulties for researchers or engineers and prevents the application of MARL to a broader range of real-world problems. To address this issue, in this manuscript we describe a framework, referred to as TLeague, that aims at large-scale training and implements several main-stream CSP-MARL algorithms. The training can be deployed in either a single machine or a cluster of hybrid machines (CPUs and GPUs), where the standard Kubernetes is supported in a cloud native manner. TLeague achieves a high throughput and a reasonable scale-up when performing distributed training. Thanks to the modular design, it is also easy to extend for solving other multi-agent problems or implementing and verifying MARL algorithms. We present experiments over StarCraft II, ViZDoom and Pommerman to show the efficiency and effectiveness of TLeague. The code is open-sourced and available at https://github.com/tencent-ailab/tleague_projpage

show abstract

“…For example, Ivanovic et al (2018) propose to create a backwards curriculum for continuous control tasks through learning a dynamics model. Resnick et al (2018b) andSalimans &Chen (2018) propose to train policies on Pommerman (Resnick et al, 2018a) and the Atari game 'Montezumas Revenge' by starting each episode from a different point along a demonstration. Recently, Goyal et al (2018) and propose a learned backtracking model to generate traces that lead to high value states in order to obtain higher sample efficiency.…”

Section: Related Workmentioning

confidence: 99%

Competitive Experience Replay

Liu,

Trott,

Socher

et al. 2019

Preprint

View full text Add to dashboard Cite

Deep learning has achieved remarkable successes in solving challenging reinforcement learning (RL) problems when dense reward function is provided. However, in sparse reward environment it still often suffers from the need to carefully shape reward function to guide policy optimization. This limits the applicability of RL in the real world since both reinforcement learning and domain-specific knowledge are required. It is therefore of great practical importance to develop algorithms which can learn from a binary signal indicating successful task completion or other unshaped, sparse reward signals. We propose a novel method called competitive experience replay, which efficiently supplements a sparse reward by placing learning in the context of an exploration competition between a pair of agents. Our method complements the recently proposed hindsight experience replay (HER) by inducing an automatic exploratory curriculum. We evaluate our approach on the tasks of reaching various goal locations in an ant maze and manipulating objects with a robotic arm. Each task provides only binary rewards indicating whether or not the goal is achieved. Our method asymmetrically augments these sparse rewards for a pair of agents each learning the same task, creating a competitive game designed to drive exploration. Extensive experiments demonstrate that this method leads to faster converge and improved task performance.

show abstract

Pommerman: A Multi-Agent Playground

Cited by 27 publications

References 0 publications

Collective Intelligence for Deep Learning: A Survey of Recent Developments

Collective Intelligence for Deep Learning: A Survey of Recent Developments

TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning

Competitive Experience Replay

Contact Info

Product

Resources

About