Using an agent-based approach for robust automated testing of computer games

Abstract

Modern computer games typically have a huge interaction spaces and non-deterministic environments. Automation in testing can provide a vital boost in development and it further improves the overall software's reliability and efficiency. Moreover, layout and game logic may regularly change during development or consecutive releases which makes it difficult to test because the usage of the system continuously changes. To deal with the latter, tests also need to be robust. Unfortunately, existing game testing approaches are not capable of maintaining test robustness. To address these challenges, this paper presents an agent-based approach for robust automated testing based on the reasoning type of AI. CCS CONCEPTS• Software and its engineering → Software testing and debugging; Interactive games.

An agent-based approach to automated game testing: an experience report

“…The iv4xr framework tries to fill this role. The advantage of a more programming-based approach is that we have much more control on the test agent behavior, and we have also shown that a BDI test agent is robust against development time changes [15].…”

“…For more combinators, including conditional and repetition, see [13], with which even a test algorithm can be expressed, e.g. when the exact sequence of sub-tasks is not known upfront [15].…”

“…This would make the test less robust. Instead we use an implementation of the online search algorithm in [15] to let the agent autonomously do the search. The algorithm will require some components to be 'plugged-in', such as the goal This constructs a goal structure for an agent 𝑎, that seeks to change the state of object 𝑜 to a new state satisfying 𝜙.…”

Learning to Identify Perceptual Bugs in 3D Video Games

“…Recent works on explorative agents for ABD have developed more sophisticated agents that are closer to human testers [1] than the more traditional rule/search based systems [19,3,16]. The incorporation of learning into these agents has become a key tool in allowing them to explore more effectively.…”

“…We believe this is due to the problem's inherent difficulty, but also at least in part due to the difficulty of the experimental set up, as can be seen in [11]. In addition, the lack of a shared platform, or comparative baselines for testing ABD approaches has led to a somewhat fractured literature and to difficulties in evaluating and comparing performances, as can be seen on the exploration side of ABD [2,4,16,19].…”