State-of-the-Art and Open Challenges in RTS Game-AI and Starcraft

Khan, Adil; Jiang, Feng; Liu, Shaohui; Jifara, Worku; Tian, Zhihong; Fu, Yunsheng

doi:10.14569/ijacsa.2017.081203

Cited by 7 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As computers advanced in the early 1940s, programmers created novel virtual worlds and unexpected human-computer interaction methods. Later on, thanks to technological developments like GPUs [1] and TPUs [2] and the innovation in neural networks [3], It is possible to use artificial intelligence significantly in games [4]. Because of this, Atari games have used reinforcement learning algorithms and strategies to train agents.…”

Section: Introductionmentioning

confidence: 99%

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Khan,

Shah,

Khan

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Advances in deep reinforcement learning have made it possible to create artificial intelligencebased agents for games that use visual information to make decisions as accurately as humans. Novel procedures are often evaluated in two-dimensional games. However, they are relatively easy compared to three-dimensional games, which have a significantly larger state and action space and, more prominently, contain partially observable states. Thus, this paper trains agents with different reinforcement learning algorithms that work fine in contradiction of human players and in-built agents by evaluating them in the first-person shooter (FPS) game Doom using the VizDoom platform. The agents learned in three different scenarios (maps): ' Defend the Center,' 'Deadly Corridor,' and 'Health gathering.' C51-DDQN, DFP, and REINFORCE algorithms have been proven effective in this study. To assess how well the trained agents performed using various reinforcement learning algorithms, we compared the results of our research with other findings in the literature. Finally, this paper presents a comparative analysis and future research directions.

show abstract

Section: Introductionmentioning

confidence: 99%

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Khan,

Shah,

Khan

et al. 2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…However, in more complex tasks, such as real-time strategy (RTS) games, RL suffers from long training times, requiring hundreds of millions of training steps [24], which can take up to weeks to complete depending on the performance of the hardware. Due to the incredibly large state and action spaces and sparse reward nature of such tasks, vanilla RL agents with no further optimisations are incapable of mastering RTS games within a reasonable time [2]. As a result, shortening the training time has the potential to significantly benefit RL agents and the outcomes in complex environments.…”

Section: Introductionmentioning

confidence: 99%

Curriculum Generation and Sequencing for Deep Reinforcement Learning in StarCraft II

Hao

Sweetser

Aitchison

2022

Australasian Computer Science Week 2022

View full text Add to dashboard Cite

Reinforcement learning has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model's ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution for reinforcement learning. Due to limitations involved with automating curriculum learning, curricula are usually manually designed. However, due to a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for reinforcement learning in real-time strategy game, StarCraft II. We propose three generalised methods of manually creating tasks for curriculum learning and verify their effectiveness through experiments. We also experiment with different curricula sequences, in addition to the most commonly used easy-to-hard order. Our results show that all three of our proposed methods can improve a reinforcement learning agent's learning process when used correctly. We demonstrate that modifying the state space of the tasks is the most effective way to create training samples for StarCraft II and that reversed curricula can be beneficial to an agent's convergence process under certain circumstances. CCS CONCEPTS• Computing methodologies → Reinforcement learning; • Software and its engineering → Interactive games; • Applied computing → Computer games.

show abstract

“…However, in more complex tasks, such as real-time strategy (RTS) games, RL suffers from long training times, requiring hundreds of millions of training steps [20], which can take up to weeks to complete depending on the performance of the hardware. Due to the incredibly large environment and sparse reward nature of such tasks, RL agents with no further optimisations are incapable of mastering RTS games within a reasonable time [2]. As a result, shortening the training time has the potential to significantly benefit RL agents and the outcomes in complex environments.…”

Section: Introductionmentioning

confidence: 99%

Designing Curriculum for Deep Reinforcement Learning in StarCraft II

Hao

Sweetser

Aitchison

2020

AI 2020: Advances in Artificial Intelligence

View full text Add to dashboard Cite

Reinforcement learning (RL) has proven successful in games, but suffers from long training times when compared to other forms of machine learning. Curriculum learning, an optimisation technique that improves a model's ability to learn by presenting training samples in a meaningful order, known as curricula, could offer a solution. Curricula are usually designed manually, due to limitations involved with automating curricula generation. However, as there is a lack of research into effective design of curricula, researchers often rely on intuition and the resulting performance can vary. In this paper, we explore different ways of manually designing curricula for RL in real-time strategy game StarCraft II. We propose four generalised methods of manually creating curricula and verify their effectiveness through experiments. Our results show that all four of our proposed methods can improve a RL agent's learning process when used correctly. We demonstrate that using subtasks, or modifying the state space of the tasks, is the most effective way to create training samples for StarCraft II. We found that utilising subtasks during training consistently accelerated the learning process of the agent and improved the agent's final performance.

show abstract

State-of-the-Art and Open Challenges in RTS Game-AI and Starcraft

Cited by 7 publications

References 19 publications

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Using VizDoom Research Platform Scenarios for Benchmarking Reinforcement Learning Algorithms in First-Person Shooter Games

Curriculum Generation and Sequencing for Deep Reinforcement Learning in StarCraft II

Designing Curriculum for Deep Reinforcement Learning in StarCraft II

Contact Info

Product

Resources

About