How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

Ammanabrolu, Prithviraj; Urbanek, Jack; Li, Margaret; Szlam, Arthur; Rocktäschel, Tim; Weston, Jason

doi:10.18653/v1/2021.naacl-main.64

Cited by 18 publications

(18 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous works also considered identifying a task by textual goal specifications (Bahdanau et al, 2019;Fu et al, 2019). In the domain of text-based games, such goal-conditioned RL setting has been studied with the quest generation tasks (Ammanabrolu et al, 2019(Ammanabrolu et al, , 2021. In our work, we specify a subtask by its goal.…”

Section: Hierarchical Reinforcement Learningmentioning

confidence: 99%

Generalization in Text-based Games via Hierarchical Reinforcement Learning

Xu¹,

Fang²,

Chen³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Deep reinforcement learning provides a promising approach for text-based games in studying natural language communication between humans and artificial agents. However, the generalization still remains a big challenge as the agents depend critically on the complexity and variety of training tasks. In this paper, we address this problem by introducing a hierarchical framework built upon the knowledge graph-based RL agent. In the high level, a meta-policy is executed to decompose the whole game into a set of subtasks specified by textual goals, and select one of them based on the KG. Then a subpolicy in the low level is executed to conduct goal-conditioned reinforcement learning. We carry out experiments on games with various difficulty levels and show that the proposed method enjoys favorable generalizability.

show abstract

Section: Hierarchical Reinforcement Learningmentioning

confidence: 99%

Generalization in Text-based Games via Hierarchical Reinforcement Learning

Xu¹,

Fang²,

Chen³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

show abstract

“…The authors design LIGHT to investigate how 'a model can both speak and act grounded in perception of its environment and dialogue from other speakers'. Ammanabrolu et al [2020b] extended this by providing a system that incorporates '1) large-scale language modelling based commonsense reasoning pre-training to imbue the agent with relevant priors and 2) a factorized action space of commands and dialogue'. Furthermore, evaluation can be performed against a dataset collected of held-out human demonstrations.…”

Section: Understanding Parser Feedback and Language Acquisitionmentioning

confidence: 99%

A Survey of Text Games for Reinforcement Learning informed by Natural Language

Osborne¹,

Nõmm²,

Freitas³

2021

Preprint

View full text Add to dashboard Cite

Reinforcement Learning has shown success in a number of complex virtual environments. However, many challenges still exist towards solving problems with natural language as a core component. Interactive Fiction Games (or Text Games) are one such problem type that offer a set of partially observable environments where natural language is required as part of the reinforcement learning solutions. Therefore, this survey's aim is to assist in the development of new Text Game problem settings and solutions for Reinforcement Learning informed by natural language. Specifically, this survey summarises: 1) the challenges introduced in Text Game Reinforcement Learning problems, 2) the generation tools for evaluating Text Games and the subsequent environments generated and, 3) the agent architectures currently applied are compared to provide a systematic review of benchmark methodologies and opportunities for future researchers.

show abstract

“…Benchmarks on embodied agents and language Benchmarks featuring language and agents embodied in physical worlds already exists, however many of them only consider the aforementioned instruction-following [Chevalier-Boisvert et al, 2018a, Misra et al, 2018, Ruis et al, 2020 and question-answering [Das et al, 2017, Gordon et al, 2018 scenarios. In between disembodied NLP testbeds [Wang et al, 2018, Zadeh et al, 2019 and previous embodied benchmarks is the LIGHT environment [Urbanek et al, 2019], a multiplayer text adventure game allowing to study social settings requiring complex dialogue production [Ammanabrolu et al, 2020, Prabhumoye et al, 2020. Instead of the virtual embodiment of text-worlds, SocialAI tackles the arguably harder and richer setting of egocentric embodiment among embodied social peers.…”

Section: Related Workmentioning

confidence: 99%

“…In the last few years however, recent advances in both DRL and NLP made the Machine Learning community reconsider experiments with language based interactions [Luketina et al, 2019, Bender andKoller, 2020]. Text-based exploratory games have been leveraged to study the capacities of autonomous agents to properly navigate through language in abstract worlds [Côté et al, 2018, Prabhumoye et al, 2020, Ammanabrolu et al, 2020. While these environments allow meaningful abstractions, they neglect the importance of embodiment for language learning, which has long been identified as an essential component for proper language understanding and grounding [Cangelosi et al, 2010, Bisk et al, 2020.…”

Section: Introductionmentioning

confidence: 99%

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

Kovač,

Portelas,

Hofmann

et al. 2021

Preprint

View full text Add to dashboard Cite

Building embodied autonomous agents capable of participating in social interactions with humans is one of the main challenges in AI. Within the Deep Reinforcement Learning (DRL) field, this objective motivated multiple works on embodied language use. However, current approaches focus on language as a communication tool in very simplified and non-diverse social situations: the "naturalness" of language is reduced to the concept of high vocabulary size and variability. In this paper, we argue that aiming towards human-level AI requires a broader set of key social skills: 1) language use in complex and variable social contexts; 2) beyond language, complex embodied communication in multimodal settings within constantly evolving social worlds. We explain how concepts from cognitive sciences could help AI to draw a roadmap towards human-like intelligence, with a focus on its social dimensions. As a first step, we propose to expand current research to a broader set of core social skills. To do this, we present SocialAI, a benchmark to assess the acquisition of social skills of DRL agents using multiple grid-world environments featuring other (scripted) social agents. We then study the limits of a recent SOTA DRL approach when tested on SocialAI and discuss important next steps towards proficient social agents. Videos and code are available at https://sites.google.com/view/socialai.

show abstract

How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds

Cited by 18 publications

References 40 publications

Generalization in Text-based Games via Hierarchical Reinforcement Learning

Generalization in Text-based Games via Hierarchical Reinforcement Learning

A Survey of Text Games for Reinforcement Learning informed by Natural Language

SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agents

Contact Info

Product

Resources

About