Efficient Reinforcement Learning for <i>StarCraft</i> by Abstract Forward Models and Transfer Learning

Liu, Ruo-Ze; Guo, Haifeng; Ji, Xiaozhong; Yu, Yang; Pang, Zhen-Jia; Xiao, Zitai; Wu, Yuezhou; Lü, Tong

doi:10.1109/tg.2021.3071162

Cited by 9 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By assembling various algorithms as modules into the framework, the efficiency and simplicity of the integrated reinforcement learning framework can be greatly improved. Furthermore, there exist integration relationships among some reinforcement learning methods [57], where certain algorithms can serve as submodules within other algorithms [58], providing a solid foundation for implementing the integrated framework. In summary, an integrated framework for reinforcement learning involves organically integrating multiple modules (different types of intelligent methods) into a versatile composite intelligent system.…”

Section: Framework Design Ideasmentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

Extensive research has been carried out on reinforcement learning methods. The core idea of reinforcement learning is to learn methods by means of trial and error, and it has been successfully applied to robotics, autonomous driving, gaming, healthcare, resource management, and other fields. However, when building reinforcement learning solutions at the edge, not only are there the challenges of data-hungry and insufficient computational resources but also there is the difficulty of a single reinforcement learning method to meet the requirements of the model in terms of efficiency, generalization, robustness, and so on. These solutions rely on expert knowledge for the design of edge-side integrated reinforcement learning methods, and they lack high-level system architecture design to support their wider generalization and application. Therefore, in this paper, instead of surveying reinforcement learning systems, we survey the most commonly used options for each part of the architecture from the point of view of integrated application. We present the characteristics of traditional reinforcement learning in several aspects and design a corresponding integration framework based on them. In this process, we show a complete primer on the design of reinforcement learning architectures while also demonstrating the flexibility of the various parts of the architecture to be adapted to the characteristics of different edge tasks. Overall, reinforcement learning has become an important tool in intelligent decision making, but it still faces many challenges in the practical application in edge computing. The aim of this paper is to provide researchers and practitioners with a new, integrated perspective to better understand and apply reinforcement learning in edge decision-making tasks.

show abstract

Section: Framework Design Ideasmentioning

confidence: 99%

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Wu,

Zhang,

Miao

et al. 2024

Electronics

View full text Add to dashboard Cite

show abstract

“…Reinforcement learning (RL), as a powerful tool for sequential decision-making, has achieved remarkable successes in a number of challenging tasks varying from board games [1], arcade games [2], robot control [3], scheduling problems [4] to autonomous driving [5]. Despite that RL algorithms have been widely assessed on game benchmarks (e.g., Atari games [6], ViZDoom [7], and DeepMind Lab [8]), the applications of RL in commercial games (e.g., StarCraft I [9] & II [10],…”

Section: Introductionmentioning

confidence: 99%

Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning Toward Asynchronous Commercial Games

et al. 2024

View full text Add to dashboard Cite

Despite the emerging progress of integrating evolutionary computation into reinforcement learning, the absence of a high-performance platform endowing composability and massive parallelism causes non-trivial difficulties for research and applications related to asynchronous commercial games. Here we introduce Lamarckian 1 -an open-source platform featuring support for evolutionary reinforcement learning scalable to distributed computing resources. To improve the training speed and data efficiency, Lamarckian adopts optimized communication methods and an asynchronous evolutionary reinforcement learning workflow. To meet the demand for an asynchronous interface by commercial games and various methods, Lamarckian tailors an asynchronous Markov Decision Process interface and designs an object-oriented software architecture with decoupled modules. In comparison with the state-of-the-art RLlib, we empirically demonstrate the unique advantages of Lamarckian on benchmark tests with up to 6000 CPU cores: i) both the sampling efficiency and training speed are doubled when running PPO on Google football game; ii) the training speed is 13 times faster when running PBT+PPO on Pong game. Moreover, we also present two use cases: i) how Lamarckian is applied to generating behavior-diverse game AI; ii) how Lamarckian is applied to game balancing tests for an asynchronous commercial game.

show abstract

Semantic Collaboration for Multi-agent: Theory, Framework, and Prospects

Li,

Qin,

Cao

2024

Lecture Notes in Electrical Engineering

View full text Add to dashboard Cite

Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning

Cited by 9 publications

References 25 publications

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

How to Design Reinforcement Learning Methods for the Edge: An Integrated Approach toward Intelligent Decision Making

Lamarckian Platform: Pushing the Boundaries of Evolutionary Reinforcement Learning Toward Asynchronous Commercial Games

Semantic Collaboration for Multi-agent: Theory, Framework, and Prospects

Contact Info

Product

Resources

About