Go-Blend Behavior and Affect

Proceedings of the 17th International Conference on the Foundations of Digital Games

et al. 2022

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of the most promising methods for achieving that long-standing goal is the use of generative AI agents, namely procedural personas, that attempt to imitate particular playing behaviors which are represented as rules, rewards, or human demonstrations. All research efforts for building those generative agents, however, have focused solely on playing behavior which is arguably a narrow perspective of what a player actually does in a game. Motivated by this gap in the existing state of the art, in this paper we extend the notion of behavioral procedural personas to cater for player experience, thus examining generative agents that can both behave and experience their game as humans would. For that purpose, we employ the Go-Explore reinforcement learning paradigm for training human-like procedural personas, and we test our method on behavior and experience demonstrations of more than 100 players of a racing game. Our findings suggest that the generated agents exhibit distinctive play styles and experience responses of the human personas they were designed to imitate. Importantly, it also appears that experience, which is tied to playing behavior, can be a highly informative driver for better behavioral exploration. CCS CONCEPTS• Applied computing → Computer games; • Computing methodologies → Reinforcement learning.

Section: Introductionmentioning

confidence: 86%

Section: Reinforcement Learning and Go-explorementioning

confidence: 99%

Section: Go-blend For Procedural Personasmentioning

confidence: 99%

Section: Algorithmmentioning

confidence: 99%

See 2 more Smart Citations

Generative Personas That Behave and Experience Like Humans

Matthew

Khalifa

Proceedings of the 17th International Conference on the Foundations of Digital Games

et al. 2022

“…Extensive work [2,3,13] in dissimilar domains of AI and games such as player experience modeling, general gameplaying or content generation make use of the internal state of the game [1,9] obtained from the game engine. Using computer vision to obtain such state information from on-screen game footage, instead of directly from the game engine, remains challenging [11].…”

Section: Introductionmentioning

confidence: 99%

Game State Learning via Game Scene Augmentation

Trivedi

Makantasis

Proceedings of the 17th International Conference on the Foundations of Digital Games

et al. 2022

Self Cite

Having access to accurate game state information is of utmost importance for any artificial intelligence task including game-playing, testing, player modeling, and procedural content generation. Self-Supervised Learning (SSL) techniques have shown to be capable of inferring accurate game state information from the highdimensional pixel input of game footage into compressed latent representations. Contrastive Learning is a popular SSL paradigm where the visual understanding of the game's images comes from contrasting dissimilar and similar game states defined by simple image augmentation methods. In this study, we introduce a new game scene augmentation technique-named GameCLR-that takes advantage of the game-engine to define and synthesize specific, highlycontrolled renderings of different game states, thereby, boosting contrastive learning performance. We test our GameCLR technique on images of the CARLA driving simulator environment and compare it against the popular SimCLR baseline SSL method. Our results suggest that GameCLR can infer the game's state information from game footage more accurately compared to the baseline. Our proposed approach allows us to conduct game artificial intelligence research by directly utilizing screen pixels as input. CCS CONCEPTS• Computing methodologies → Learning latent representations; Visual content-based indexing and retrieval; • Applied computing → Computer games.

Play with Emotion: Affect-Driven Reinforcement Learning

Matthew

Khalifa

2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII)

et al. 2022

Self Cite

This paper introduces a paradigm shift by viewing the task of affect modeling as a reinforcement learning (RL) process. According to the proposed paradigm, RL agents learn a policy (i.e. affective interaction) by attempting to maximize a set of rewards (i.e. behavioral and affective patterns) via their experience with their environment (i.e. context). Our hypothesis is that RL is an effective paradigm for interweaving affect elicitation and manifestation with behavioral and affective demonstrations. Importantly, our second hypothesis-building on Damasio's somatic marker hypothesis-is that emotion can be the facilitator of decision-making. We test our hypotheses in a racing game by training Go-Blend agents to model human demonstrations of arousal and behavior; Go-Blend is a modified version of the Go-Explore algorithm which has recently showcased supreme performance in hard exploration tasks. We first vary the arousalbased reward function and observe agents that can effectively display a palette of affect and behavioral patterns according to the specified reward. Then we use arousal-based state selection mechanisms in order to bias the strategies that Go-Blend explores. Our findings suggest that Go-Blend not only is an efficient affect modeling paradigm but, more importantly, affect-driven RL improves exploration and yields higher performing agents, validating Damasio's hypothesis in the domain of games.