Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models

Zhao, Tiancheng; Xie, Kaige; Eskénazi, Maxine

doi:10.48550/arxiv.1902.08858

Cited by 18 publications

(29 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Though these methods have achieved promising results, they were usually designed for a specific domain, rendering difficulties in generalizing to multi-domains, e.g., the recently proposed multi-domain dataset MultiWoz (Eric et al 2019). Subsequently, there are several models are proposed to handle the multi-domain response generation task (Zhao, Xie, and Eskenazi 2019;Chen et al 2019;Qin et al 2020). To prevent dialog acts growing combinatorially with the number of domains, Chen et al ( 2019) built a multi-layer hierarchical graph to represent dialog acts to generate responses using BERT-based dialog policy.…”

Section: Related Workmentioning

confidence: 99%

A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining

Zhang¹,

Lyu²,

Ding³

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper describes our submission for the End-to-end Multi-domain Task Completion Dialog shared task at the 9th Dialog System Technology Challenge (DSTC-9). Participants in the shared task build an end-to-end task completion dialog system which is evaluated by human evaluation and a user simulator based automatic evaluation. Different from traditional pipelined approaches where modules are optimized individually and suffer from cascading failure, we propose an end-to-end dialog system that 1) uses Generative Pretraining 2 (GPT-2) as the backbone to jointly solve Natural Language Understanding, Dialog State Tracking, and Natural Language Generation tasks, 2) adopts Domain and Task Adaptive Pretraining to tailor GPT-2 to the dialog domain before finetuning, 3) utilizes heuristic pre/post-processing rules that greatly simplify the prediction tasks and improve generalizability, and 4) equips a fault tolerance module to correct errors and inappropriate responses. Our proposed method significantly outperforms baselines and ties for first place in the official evaluation. We make our source code publicly available.* Boliang Zhang and Ying Lyu contributed equally to this work.

show abstract

Section: Related Workmentioning

confidence: 99%

A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining

Zhang¹,

Lyu²,

Ding³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…One drawback of end-to-end neural systems is that training the word-level policy with RL is very difficult due to the large action space. To mitigate this issue, Zhao et al (2019) proposed to learn policy networks in the latent action space. proposed a universal and scalable belief tracker by jointly learning the NLU and DST modules, improving the flexibility of domain ontology configurations.…”

Section: End-to-end Neural Dialog Systemmentioning

confidence: 99%

“…Succ. TSCP (Lei et al, 2018) 18.20 32.0 13.68 11.8 SUMBT 13.71 44.0 46.44 27.8 LaRL (Zhao et al, 2019) 13.08 68.0 68.95 47.7 DAMD (Zhang et al, 2020a) 11.27 64.0 59.7 48.5 M-GDPL (Takanobu et al, 2019) Table 1: System-wise automatic evaluation performance of dialog turns, inform recall, match rate, and success rate, for all baseline systems and our S-PPO system. S-PPO beats all baselines and achieves state-of-the-art results.…”

Section: Trainingmentioning

confidence: 99%

See 1 more Smart Citation

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

Lin¹,

Huang²,

Zhou³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent work proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e.g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation. To improve the system-wise performance, in this paper, we propose new joint system-wise optimization techniques for the pipeline dialog system. First, we propose a new data augmentation approach which automates the labeling process for NLU training. Second, we propose a novel stochastic policy parameterization with Poisson distribution that enables better exploration and offers a principled way to compute policy gradient. Third, we propose a reward bonus to help policy explore successful dialogs. Our approaches outperform the competitive pipeline systems from Takanobu et al. ( 2020) by big margins of 12% success rate in automatic system-wise evaluation and of 16% success rate in human evaluation on the standard multi-domain benchmark dataset MultiWOZ, and also outperform the recent state-of-the-art end-to-end trained model from DSTC9 (Gunasekara et al., 2020).

show abstract

“…And whether a model can generate diverse (Xu et al, 2018;Baheti et al, 2018), coherent (Li et al, 2016bTian et al, 2017;Bosselut et al, 2018;Adiwardana et al, 2020), informative (Shao et al, 2017;Lewis et al, 2017;Ghazvininejad et al, 2017;Young et al, 2017;Zhao et al, 2019) and knowledge-fused (Hua et al, 2020;Zhao et al, 2020;He et al, 2020) responses or not has become metrics to evaluate a dialog generation model. However, the mainly researches described above are developed on textual only and the development of multimodal dialog generation is relatively slow since the lack of large-scale datasets.…”

Section: Dialog Generationmentioning

confidence: 99%

OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts

Wang¹,

Meng²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

In order to better simulate the real human conversation process, models need to generate dialogue utterances based on not only preceding textual contexts but also visual contexts. However, with the development of multi-modal dialogue learning, the dataset scale gradually becomes a bottleneck. In this report, we release OpenViDial 2.0, a larger-scale open-domain multi-modal dialogue dataset compared to the previous version OpenViDial 1.0 (Meng et al., 2020). OpenViDial 2.0 contains a total number of 5.6 million dialogue turns extracted from either movies or TV series from different resources, and each dialogue turn is paired with its corresponding visual context. We hope this large-scale dataset can help facilitate future researches on open-domain multi-modal dialog generation, e.g., multi-modal pretraining for dialogue generation. 1

show abstract

Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models

Cited by 18 publications

References 30 publications

A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining

A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining

Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts

Contact Info

Product

Resources

About