Curriculum Generation and Sequencing for Deep Reinforcement Learning in StarCraft II

Abstract

Reinforcement learning has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model's ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution for reinforcement learning. Due to limitations involved with automating curriculum learning, curricula are usually manually designed. However, due to a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for reinforcement learning in real-time strategy game, StarCraft II. We propose three generalised methods of manually creating tasks for curriculum learning and verify their effectiveness through experiments. We also experiment with different curricula sequences, in addition to the most commonly used easy-to-hard order. Our results show that all three of our proposed methods can improve a reinforcement learning agent's learning process when used correctly. We demonstrate that modifying the state space of the tasks is the most effective way to create training samples for StarCraft II and that reversed curricula can be beneficial to an agent's convergence process under certain circumstances. CCS CONCEPTS• Computing methodologies → Reinforcement learning; • Software and its engineering → Interactive games; • Applied computing → Computer games.

Leveraging quantum‐inspired chimp optimization and deep neural networks for enhanced profit forecasting in financial accounting systems

“…In games like GO and StarCraft II, where they must overcome challenging reward-maximizing competitive problems, environmental simulators have shown themselves capable of competing with humans (Hao et al, 2022). They have also shown themselves to be excellent learning environments for DL approaches.…”