Interactively evolved modular neural networks for game agent control

Abstract

Abstract-As the realism in games continues to increase, through improvements in graphics and 3D engines, more focus is placed on the behavior of the simulated agents that inhabit the simulated worlds. The agents in modern video games must become more life-like in order to seem to belong in the environments they are portrayed in. Many modern artificial intelligence approaches achieve a high level of realism but this is accomplished through significant developer time spent scripting the behaviors of the Non-Playable Characters or NPC's. These agents will behave in a believable fashion in the scenarios they have been programmed for, but do not have the ability to adapt to new situations. In this paper we introduce a modularized, real-time evolution training technique to evolve adaptable agents with life-like behaviors. Online performance during evolution is also improved by using selection mechanisms found in temporal difference learning methods to appropriately balance the exploration and exploitation of control policies. These methods are implemented and tested using the XNA framework producing very promising results regarding efficiency of techniques, and demonstrating many potential avenues for further research.

Neuroevolution in Games: State of the Art and Open Challenges

“…Previous research by Whiteson and Stone in combining NEAT and Q-learning has shown promising results [138,139]. These methods have been applied with some success to shooters [89] and racing games [13].…”

Genetic Representation and Evolvability of Modular Neural Controllers

“…However, when the effect of mutations was constrained by a hand-designed modular network structure separating the two subproblems, evolution was able to find optimal neural controllers consistently. In the literature there are other examples where researchers relied on manual decomposition of the control problem by subdividing the control architectures into a predefined number of modules, based on a priori knowledge of the problem (e.g., [18], [19], [20]). However, it is not always obvious how control problems should be decomposed into tractable sub-problems [21].…”

The Amalthea Reu Program: Activities, Experiences, And Outcomes Of A Collaborative Summer Research Experience In Machine Learning

“…Additionally, it has partially/indirectly supported an additional Masters thesis 21 . The REU research has also led to the publication with former undergraduate participants of 5 conference papers [22][23][24][25][26] (with undergraduates Hoover, Beck, Garcia, Cardona, Alexander, Ahmed, Sparks and Miguez) and 1 journal paper 27 (with undergraduate Hoover). Also, it has indirectly and subsequently aided in the research presented in 9 additional conferences and 2 journals, which we do not list here.…”