Generative Personas That Behave and Experience Like Humans

Abstract

Using artificial intelligence (AI) to automatically test a game remains a critical challenge for the development of richer and more complex game worlds and for the advancement of AI at large. One of the most promising methods for achieving that long-standing goal is the use of generative AI agents, namely procedural personas, that attempt to imitate particular playing behaviors which are represented as rules, rewards, or human demonstrations. All research efforts for building those generative agents, however, have focused solely on playing behavior which is arguably a narrow perspective of what a player actually does in a game. Motivated by this gap in the existing state of the art, in this paper we extend the notion of behavioral procedural personas to cater for player experience, thus examining generative agents that can both behave and experience their game as humans would. For that purpose, we employ the Go-Explore reinforcement learning paradigm for training human-like procedural personas, and we test our method on behavior and experience demonstrations of more than 100 players of a racing game. Our findings suggest that the generated agents exhibit distinctive play styles and experience responses of the human personas they were designed to imitate. Importantly, it also appears that experience, which is tied to playing behavior, can be a highly informative driver for better behavioral exploration. CCS CONCEPTS• Applied computing → Computer games; • Computing methodologies → Reinforcement learning.

Play with Emotion: Affect-Driven Reinforcement Learning

“…Interestingly, modeling arousal based on the expert players' annotations led to high-performing agents despite the fact that "winning" the game was not explicitly targeted by the algorithm. Along with [35], this is the first application of the Go-Blend algorithm in a complex and affect-intense game, opening many directions for future work at the intersection of affect modeling, RL and affect-driven machine learning.…”

“…Interestingly, imitating these players' arousal traces biased the agents towards correct in-game strategies. Experiments with low performing players [35] did not result in competent agents when imitating arousal alone, which means that conclusions regarding the power of arousal as a reward or strategy in RL tasks pre-supposes that high-quality arousal and performance examples exist. On the other hand, the fact that only a small portion of the population is considered may explain why reward based on consensus between arousal traces (R au , R ac ) did not perform as well.…”

“…Note that due to the large size of the dataset, and a significant gap in performance between players, we used hierarchical clustering on players' aggregated session data to identify the best performing group. Details on the clustering process and experiments with Go-Blend on different player groups are provided in [35]. The result of this process is a dataset of 27 "expert" players with similar behavioral characteristics.…”

Evolutionary Machine Learning and Games

“…Migkotzidis and Liapis [65] used the same model as part of a mixed-initiative tool [116] that can be used to design balanced levels for FPS games. On the side of affective computing, Barthet et al [4] use a variant of Go-Explore called Go-Blend (a quality diversity algorithm that keeps track of the best solutions) to explore game trajectories that can mimic different human play styles and arousal levels. They used a simple K-NN algorithm over the AGAIN dataset [64] to predict the arousal levels during playing a car racing game.…”

Individuality and the collective in AI agents: Explorations of shared consciousness and digital homunculi in the metaverse for cultural heritage

“…Concurrently, Holmgard et al [43] postulated that artificial agents might serve as psychometrically accurate abstract simulations of the internal decision-making procedures of human players. Similarly, Barthet et al [44] engineered generative personas that reflected human-like behavior, manifesting play styles and reactions that faithfully paralleled the human models they sought to emulate.…”