Learning a Super Mario controller from examples of human play

Abstract

Imitating human-like behaviour in action games is a challenging but intriguing task in Artificial Intelligence research, with various strategies being employed to solve the human-like imitation problem. In this research we consider learning human-like behaviour via Markov decision processes without being explicitly given a reward function, and learning to perform the task by observing expert's demonstration. Individual players often have characteristic styles when playing the game, and this method attempts to find the behaviours which make them unique. During play sessions of Super Mario we calculate player's behaviour policies and reward functions by applying inverse reinforcement learning to the player's actions in game. We conduct an online questionnaire which displays two video clips, where one is played by a human expert and the other is played by the designed controller based on the player's policy. We demonstrate that by using apprenticeship learning via Inverse Reinforcement Learning, we are able to get an optimal policy which yields performance close to that of an human expert playing the game, at least under specific conditions.

Persona Agents from Playtraces and Emotion

“…Another work that also uses IRL can be seen in Lee et al (Lee et al 2014) where the authors applied apprenticeship learning in order to learn a platformer game, "Super Mario", from observing players. Other applications of IRL to learn different behaviours can be found in (Lee and Popović 2010) and (Almingol, Montesano, and Lopes 2013).…”

Exploring Apprenticeship Learning for Player Modelling in Interactive Narratives

“…AL applications related to our work include a Capture The Flag commander that learns policies from encounters with enemies [9], a dungeon generation tool [23], and a Super Mario controller that learns action sequences from player demonstrations [11]. While these works apply AL via IRL techniques in a game environment, they do not focus on INs.…”

Towards Intelligent Interactive Theatre: Drama Management as a way of Handling Performance

“…Another possibility is to use unsupervised learning methods, such as Apprenticeship Learning (AL) to have the agent learn a more general behavioural pattern throughout the whole story [4]. A benefit of unsupervised learning is that there is less reliance on a human expert to dictate behaviour, but a disadvantage is that the resulting behaviour may not be of the same quality as that generated by a supervised learning method.…”