A Local Approach to Forward Model Learning: Results on the Game of Life Game

Lucas, Simon M.; Dockhorn, Alexander; Volz, Vanessa; Bamford, Chris; Gaina, Raluca D.; Bravi, Ivan; Peŕez-Liebana, Diego; Mostaghim, Sanaz; Kruse, Rudolf

doi:10.1109/cig.2019.8848002

Cited by 15 publications

(7 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, in order for the system to be a truly general game-player, it should be able to play games even when a game model is not provided. Learning forward models in the general game-playing context is an active area of research, with several impressive advances [58]- [61]. With the addition of such a module, we speculate the system could even receive games from external sources (thus not adhering to any accidental assumptions included in the building of the system) and learn how to play them.…”

Section: Discussionmentioning

confidence: 99%

Project Thyia: A Forever Gameplayer

Gaina

Lucas

Peŕez-Liebana

2019

2019 IEEE Conference on Games (CoG)

Self Cite

View full text Add to dashboard Cite

The space of Artificial Intelligence entities is dominated by conversational bots. Some of them fit in our pockets and we take them everywhere we go, or allow them to be a part of human homes. Siri, Alexa, they are recognised as present in our world. But a lot of games research is restricted to existing in the separate realm of software. We enter different worlds when playing games, but those worlds cease to exist once we quit. Similarly, AI game-players are run once on a game (or maybe for longer periods of time, in the case of learning algorithms which need some, still limited, period for training), and they cease to exist once the game ends. But what if they didn't? What if there existed artificial game-players that continuously played games, learned from their experiences and kept getting better? What if they interacted with the real world and us, humans: livestreaming games, chatting with viewers, accepting suggestions for strategies or games to play, forming opinions on popular game titles? In this paper, we introduce the vision behind a new project called Thyia, which focuses around creating a present, continuous, 'always-on', interactive game-player.

show abstract

Section: Discussionmentioning

confidence: 99%

Project Thyia: A Forever Gameplayer

Gaina

Lucas

Peŕez-Liebana

2019

2019 IEEE Conference on Games (CoG)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Naturally, this also implies that it is largely dependent upon the quality of the model that it has access to. For the situations that we are concerned in this manuscript, when a forward model is not initially available, RHE has been able to interoperate with approximated or imperfect forward models (Lucas et al 2019a;Ovalle and Lucas 2020a,b;Olesen et al 2020)…”

Section: Rolling Horizon Evolutionmentioning

confidence: 99%

Predictive Control Using Learned State Space Models via Rolling Horizon Evolution

Ovalle,

Lucas

2021

Preprint

Self Cite

View full text Add to dashboard Cite

A large part of the interest in model-based reinforcement learning derives from the potential utility to acquire a forward model capable of strategic long term decision making. Assuming that an agent succeeds in learning a useful predictive model, it still requires a mechanism to harness it to generate and select among competing simulated plans. In this paper, we explore this theme combining evolutionary algorithmic planning techniques with models learned via deep learning and variational inference. We demonstrate the approach with an agent that reliably performs online planning in a set of visual navigation tasks.

show abstract

“…RHEA has proven effective across a range of games [7,5,4]. A detailed description of the agent can be found in [6]. The FM of the game is key to the evolution process (and thus the performance of the agent), as it is used to evaluate the fitness of generated solutions.…”

Section: Agent Modelmentioning

confidence: 99%

“…In previous work [3,6], we proposed to use an ensemble of local models as an FM in grid-based games. In these, the next state of a given cell often depends on the surrounding ones and thus seem specifically well suited for this approach.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Local Forward Models on Unforgiving Games

Dockhorn

Lucas

Volz

et al. 2019

2019 IEEE Conference on Games (CoG)

Self Cite

View full text Add to dashboard Cite

This paper examines learning approaches for forward models based on local cell transition functions. We provide a formal definition of local forward models for which we propose two basic learning approaches. Our analysis is based on the game Sokoban, where a wrong action can lead to an unsolvable game state. Therefore, an accurate prediction of an action's resulting state is necessary to avoid this scenario.In contrast to learning the complete state transition function, local forward models allow extracting multiple training examples from a single state transition. In this way, the Hash Set model, as well as the Decision Tree model, quickly learn to predict upcoming state transitions of both the training and the test set. Applying the model using a statistical forward planner showed that the best models can be used to satisfying degree even in cases in which the test levels have not yet been seen.Our evaluation includes an analysis of various local neighbourhood patterns and sizes to test the learners' capabilities in case too few or too many attributes are extracted, of which the latter has shown do degrade the performance of the model learner.

show abstract

A Local Approach to Forward Model Learning: Results on the Game of Life Game

Abstract: Recent work has made significant progress in learning forward models, more of which will be described in Section II. Much of this work has used Deep Neural Networks to learn forward models in the form of entire state transition functions.

Cited by 15 publications

References 14 publications

Project Thyia: A Forever Gameplayer

Project Thyia: A Forever Gameplayer

Predictive Control Using Learned State Space Models via Rolling Horizon Evolution

Learning Local Forward Models on Unforgiving Games

Contact Info

Product

Resources

About