Deep Reinforcement Learning with Transformers for Text Adventure Games

Xu, Yunqiu; Ling, Chen; Fang, Meng; Wang, Yang; Zhang, Chengqi

doi:10.1109/cog47356.2020.9231622

Cited by 22 publications

(25 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…e formula variable represents the function of distribution characteristic points. e data analysis based on the discrete situation is different from the result in formula (2).…”

Section: Research On the Technology Of Multisource Spatial Datamentioning

confidence: 95%

See 1 more Smart Citation

Urban Fine Management of Multisource Spatial Data Fusion Based on Smart City Construction

Long

Zhang²,

Gao

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

With the increase of the types of urban management objects, the intelligent management of the whole city has become a matter of concern in various countries, and it is also one of the indispensable links in urban development. In the construction of cities all over the world, the intelligent and scientific management system has been used innovatively. We provide excellent facilities for transportation development, information exchange, and resource progress. The research on urban fine management based on multisource spatial data fusion is proposed. Aiming at the traffic problems in urban fine management, this paper proposes a deep network architecture based on multisource data fusion. Multisource spatial data fusion technology is used to analyze urban traffic data. Deep network architecture is used to improve the precision management status of a smart city and the accuracy of traffic condition prediction. Then, the convolution neural network technology is explored in the data fusion technology strategy. The research results show that the framework has the ability to deal with heterogeneous data and urban big data and can effectively improve the traffic management state in the construction of a smart city and effectively solve the complexity of urban fine management and processing efficiency in the construction of a smart city.

show abstract

“…e formula variable represents the function of distribution characteristic points. e data analysis based on the discrete situation is different from the result in formula (2).…”

Section: Research On the Technology Of Multisource Spatial Datamentioning

confidence: 95%

“…It is also the primary responsibility of countries to expand urban development and demand. According to a large number of data, with the growth of the year, the resident population of each country has increased linearly [2]. e trend of quantity growth is also very obvious.…”

Section: Introductionmentioning

confidence: 99%

Urban Fine Management of Multisource Spatial Data Fusion Based on Smart City Construction

Long

Zhang²,

Gao

2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…We compare the effects of these modifications to our architecture. Lightweight transformers have shown strong performance in text adventure games [40], and the transformer encoder was applied to video games [41]. In contrast to these works, we utilize a multi-layer transformer decoder architecture.…”

Section: Related Workmentioning

confidence: 99%

Deep Transformer Q-Networks for Partially Observable Reinforcement Learning

Esslinger¹,

Platt²,

Amato³

2022

Preprint

View full text Add to dashboard Cite

Real-world reinforcement learning tasks often involve some form of partial observability where the observations only give a partial or noisy view of the true state of the world. Such tasks typically require some form of memory, where the agent has access to multiple past observations, in order to perform well. One popular way to incorporate memory is by using a recurrent neural network to access the agent's history. However, recurrent neural networks in reinforcement learning are often fragile and difficult to train, susceptible to catastrophic forgetting and sometimes fail completely as a result. In this work, we propose Deep Transformer Q-Networks (DTQN), a novel architecture utilizing transformers and self-attention to encode an agent's history. DTQN is designed modularly, and we compare results against several modifications to our base model. Our experiments demonstrate the transformer can solve partially observable tasks faster and more stably than previous recurrent approaches.Preprint. Under review.

show abstract

“…Motivated by the prosperity of deep reinforcement learning techniques in playing games , robotics (Schulman et al, 2017;Fang et al, 2019a,b) and NLP (Fang et al, 2017), several RLbased game agents have been developed for textbased games (He et al, 2016;Jain et al, 2020;Yin and May, 2019;Guo et al, 2020;Xu et al, 2020a). Compared with the nonlearning-based agents (Hausknecht et al, 2019;Atkinson et al, 2019), the RL-based agents are more favorable as there is no need to handcraft game playing strategies with huge amounts of expert knowledge.…”

Section: Rl Agent For Text-based Gamesmentioning

confidence: 99%

Generalization in Text-based Games via Hierarchical Reinforcement Learning

Xu¹,

Fang²,

Chen³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

Self Cite

View full text Add to dashboard Cite

Deep reinforcement learning provides a promising approach for text-based games in studying natural language communication between humans and artificial agents. However, the generalization still remains a big challenge as the agents depend critically on the complexity and variety of training tasks. In this paper, we address this problem by introducing a hierarchical framework built upon the knowledge graph-based RL agent. In the high level, a meta-policy is executed to decompose the whole game into a set of subtasks specified by textual goals, and select one of them based on the KG. Then a subpolicy in the low level is executed to conduct goal-conditioned reinforcement learning. We carry out experiments on games with various difficulty levels and show that the proposed method enjoys favorable generalizability.

show abstract

Deep Reinforcement Learning with Transformers for Text Adventure Games

Cited by 22 publications

References 15 publications

Urban Fine Management of Multisource Spatial Data Fusion Based on Smart City Construction

Urban Fine Management of Multisource Spatial Data Fusion Based on Smart City Construction

Deep Transformer Q-Networks for Partially Observable Reinforcement Learning

Generalization in Text-based Games via Hierarchical Reinforcement Learning

Contact Info

Product

Resources

About