Creating Pro-Level AI for a Real-Time Fighting Game Using Deep Reinforcement Learning

Abstract

Reinforcement learning combined with deep neural networks has performed remarkably well in many genres of games recently. It has surpassed human-level performance in fixed game environments and turn-based two player board games. However, to the best of our knowledge, current research has yet to produce a result that has surpassed human-level performance in modern complex fighting games. This is due to the inherent difficulties with real-time fighting games, including: vast action spaces, action dependencies, and imperfect information. We overcame these challenges and made 1v1 battle AI agents for the commercial game "Blade & Soul". The trained agents competed against five professional gamers and achieved a win rate of 62%. This paper presents a practical reinforcement learning method that includes a novel self-play curriculum and data skipping techniques. Through the curriculum, three different styles of agents were created by reward shaping and were trained against each other. Additionally, this paper suggests data skipping techniques that could increase data efficiency and facilitate explorations in vast spaces. Since our method can be generally applied to all two-player competitive games with vast action spaces, we anticipate its application to game development including level design and automated balancing.

Artificial Intelligence for the Metaverse: A Survey

“…This testing mechanism deployed a decision tree model to reveal metamorphic relations, which in turn effectively determine the optimal move among all possible ones. With a combination of RL and deep networks, AI agents in [164] were developed to address some inherent difficulties in realtime fighting games and to defeat pro players in one-vsone battle. In addition to creating different fighting styles through self-play curriculum, this deep RL framework was capable of all two-player competitive games which have level upgrade and balance policies.…”

Enhanced DQN Framework for Selecting Actions and Updating Replay Memory Considering Massive Non-Executable Actions

“…Recently, game AI models based on reinforcement learning have been developed to clear games with minimum amounts of information required to control their progress. Oh et al used deep reinforcement learning to describe a 62% real-time fighting game AI model against five professional gamers [25]. Tang et al introduced a Gomoku AI model that combines MCTS and ADP to eliminate the "short-sighted" defect of the neural network evaluation function [26].…”

Team Sports for Game AI Benchmarking Revisited

“…We can expect that more game genres will be added to this list in the nearest future. For example, a recent work by Oh et al [30] discusses the development of an AI system, able to defeat professional human players in a modern fighting game.…”