Adaptive game AI with dynamic scripting

Abstract

Online learning in commercial computer games allows computer-controlled opponents to adapt to the way the game is being played. As such it provides a mechanism to deal with weaknesses in the game AI, and to respond to changes in human player tactics. We argue that online learning of game AI should meet four computational and four functional requirements. The computational requirements are speed, effectiveness, robustness and efficiency. The functional requirements are clarity, variety, consistency and scalability. This paper investigates a novel online learning technique for game AI called 'dynamic scripting', that uses an adaptive rulebase for the generation of game AI on the fly. The performance of dynamic scripting is evaluated in experiments in which adaptive agents are pitted against a collection of manually-designed tactics in a simulated computer roleplaying game. Experimental results indicate that dynamic scripting succeeds in endowing computer-controlled opponents with adaptive performance. To further improve the dynamic-scripting technique, an enhancement is investigated that allows scaling of the difficulty level of the game AI to the human player's skill level. With the enhancement, dynamic scripting meets all computational and functional requirements. The applicability of dynamic scripting in state-of-the-art commercial games is demonstrated by implementing the technique in the game Neverwinter Nights. We conclude that dynamic scripting can be successfully applied to the online adaptation of game AI in commercial computer games.

Dynamic Scripting with Team Coordination in Air Combat Simulation

“…It was introduced by Spronck et al [7] to address certain requirements for adaptive game AI in commercial video games, such as "easily interpretable results" and "reasonably successful behavior at all times." 1 These requirements are also applicable in the domain of military training, where quality controls such as transparent results and robust behavior are important.…”

“…In the original DS experiments [7] team behavior was a result of emergence, guided by the fitness function which rewarded team victories as well as individual success. For air combat training simulations we remark that they require more control over the team members' actions, to make sure that the CGFs act conforming to the training goals of a particular simulation.…”

“…In this paper, we have chosen to implement decentralized control, because its implementation is straightforward and directly understandable. Moreover, the smallest step from having a team of one agent with one 'team rule base' is an increase to a team with two agents 1 A complete description of these requirements can be found in [7]. with two nearly similar rule bases.…”

“…Dynamic Scripting (DS) is a reinforcement learning technique specifically designed to satisfy requirements similar as the ones stated above [7]. While DS has been used with teams, no attention has been given to the explicit coordination of teams using DS.…”

Evolving explicit opponent models in game playing

“…In particular, Spronck et al [13] and Ponsen et al [10] implemented opponent modeling in Warcraft, a popular strategy game. In this domain, a form of reinforcement learning called dynamic scripting allows automated players to learn online.…”