Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning

Glavin, Frank G.; Madden, Michael G.

doi:10.1109/tciaig.2014.2363042

Cited by 26 publications

(10 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…FPS games, especially the most popular ones such as Unreal Tournament [12], [13], Counter-Strike [15] or Quake III Arena [8], have already been used in AI research. However, in these studies agents acted upon high-level information like positions of walls, enemies, locations of items, etc., which are usually inaccessible to human players.…”

Section: Introductionmentioning

confidence: 99%

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Kempka

Wydmuch

Runc

et al. 2016

2016 IEEE Conference on Computational Intelligence and Games (CIG)

471

364

View full text Add to dashboard Cite

Abstract-The recent advances in deep neural networks have led to effective vision-based reinforcement learning methods that have been employed to obtain human-level controllers in Atari 2600 games from pixel data. Atari 2600 games, however, do not resemble real-world tasks since they involve non-realistic 2D environments and the third-person perspective. Here, we propose a novel test-bed platform for reinforcement learning research from raw visual information which employs the firstperson perspective in a semi-realistic 3D world. The software, called ViZDoom, is based on the classical first-person shooter video game, Doom. It allows developing bots that play the game using the screen buffer. ViZDoom is lightweight, fast, and highly customizable via a convenient mechanism of user scenarios. In the experimental part, we test the environment by trying to learn bots for two scenarios: a basic move-and-shoot task and a more complex maze-navigation problem. Using convolutional deep neural networks with Q-learning and experience replay, for both scenarios, we were able to train competent bots, which exhibit human-like behaviors. The results confirm the utility of ViZDoom as an AI research platform and imply that visual reinforcement learning in 3D realistic first-person perspective environments is feasible.

show abstract

Section: Introductionmentioning

confidence: 99%

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Kempka

Wydmuch

Runc

et al. 2016

2016 IEEE Conference on Computational Intelligence and Games (CIG)

471

364

View full text Add to dashboard Cite

show abstract

“…Game Playing: Researchers applied RL and MCTS to numerous games, and there are plentiful studies on these topics. For example, Sarsa(λ) is used as a game playing agent in Ms. Pac Man [16] and to create a human-like agent in Unreal Tournament [12]. Although the aim of these papers was to create better agents in game playing, our purpose is to create an agent that tests the game by playing with respect to test goals.…”

Section: Related Researchmentioning

confidence: 99%

Automated Video Game Testing Using Synthetic and Humanlike Agents

2021

View full text Add to dashboard Cite

In this paper, we present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents -synthetic and human-like-and two distinct approaches to create them. Our agents are derived from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but focus on finding defects. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The human-like agent uses test goals extracted by our proposed multiple greedy-policy inverse reinforcement learning (MGP-IRL) algorithm from tester trajectories. MGP-IRL captures multiple policies executed by human testers. These testers' aims are finding defects while interacting with the game to break it, which is considerably different from game playing. We present interaction states to model such interactions. We use our agents to produce test sequences, run the game with these sequences, and check the game for each run with an automated test oracle. We analyze the proposed method in two parts: we compare the success of human-like and synthetic agents in bug finding, and we evaluate the similarity between humanlike agents and human testers. We collected 427 trajectories from human testers using the General Video Game Artificial Intelligence (GVG-AI) framework and created three games with 12 levels that contain 45 bugs. Our experiments reveal that human-like and synthetic agents compete with human testers' bug finding performances. Moreover, we show that MGP-IRL increases the human-likeness of agents while improving the bug finding performance.

show abstract

“…Our application of SEC is concerned with balancing the Assault Rifle skill of a Deathmatch NPC playing against a single opponent. The NPC is initially trained using the shooter bot implementation from Glavin and Madden [18] [19].…”

Section: Skilled Experience Cataloguementioning

confidence: 99%

Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

Glavin

Madden

2018

2018 IEEE Conference on Computational Intelligence and Games (CIG)

Self Cite

View full text Add to dashboard Cite

In this paper, we introduce a skill-balancing mechanism for adversarial non-player characters (NPCs), called Skilled Experience Catalogue (SEC). The objective of this mechanism is to approximately match the skill level of an NPC to an opponent in real-time. We test the technique in the context of a First-Person Shooter (FPS) game. Specifically, the technique adjusts a reinforcement learning NPC's proficiency with a weapon based on its current performance against an opponent. Firstly, a catalogue of experience, in the form of stored learning policies, is built up by playing a series of training games. Once the NPC has been sufficiently trained, the catalogue acts as a timeline of experience with incremental knowledge milestones in the form of stored learning policies. If the NPC is performing poorly, it can jump to a later stage in the learning timeline to be equipped with more informed decision-making. Likewise, if it is performing significantly better than the opponent, it will jump to an earlier stage. The NPC continues to learn in realtime using reinforcement learning but its policy is adjusted, as required, by loading the most suitable milestones for the current circumstances.

show abstract

Adaptive Shooting for Bots in First Person Shooter Games Using Reinforcement Learning

Cited by 26 publications

References 22 publications

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

ViZDoom: A Doom-based AI research platform for visual reinforcement learning

Automated Video Game Testing Using Synthetic and Humanlike Agents

Skilled Experience Catalogue: A Skill-Balancing Mechanism for Non-Player Characters using Reinforcement Learning

Contact Info

Product

Resources

About