Usertesting Without the User

Abstract

The use of human participants in game evaluation can be costly, time-consuming, and present challenges for constructing representative player samples. These challenges may be overcome by using computer-controlled agents in place of human users for certain stages of the usertesting process. This article explores opportunities and challenges in the use of behavioural modelling to create independent “user” agents driven by artificial intelligence (AI). We highlight the utility of imitating cognitive processes such as spatial reasoning, memory, and goal-oriented decision-making as a means to increase the viability of independent agents as a tool in usertesting. Specifically, we investigate the possible design and use of proxy AI “users” that mimic human navigational behaviour to assist in the evaluation of level designs. Ultimately, we propose that a configurable population of AI players can provide a data-rich supplement to current approaches in games user research.

Model-based Player Experience Testing with Emotion Pattern Verification

“…Ariyurek et al [7] introduces Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents to detect bugs in video games automatically. Stahlke et al [51] presents a basis for a framework to model player's memory and goal-oriented decision-making to simulate human navigational behavior for identifying level design issues. The framework creates an AI-agent that uses a path finding heuristic to navigate a level, optimized by a given player characteristics such as level of experience and play-style.…”

An Appraisal Transition System for Event-Driven Emotions in Agent-Based Player Experience Testing

“…Thus, player or set-up dependent properties must be initially set in this part of framework, before running the model of appraisal. Having such a component in our framework also provides an opportunity to enhance it in the future with more advanced characteristics such as players' moods and play-style (e.g., exploratory or aggressive [25,29]).…”

“…A variety of techniques are utilized in the provided case studies to train play-testing agents to test logic of the game under development as well as game-playing agents which interact with human players to mimic the game play experience for different play style. Stahlke et al [25] also aim to use play-testing agents to test games by following humans' navigational behavior. They investigate the impact of play-style, the experience level and cognitive process on modeling humans behavior.…”