Structured Fusion Networks for Dialog

Abstract

Neural dialog models have exhibited strong performance, however their end-to-end nature lacks a representation of the explicit structure of dialog. This results in a loss of generalizability, controllability and a datahungry nature. Conversely, more traditional dialog systems do have strong models of explicit structure. This paper introduces several approaches for explicitly incorporating structure into neural models of dialog. Structured Fusion Networks first learn neural dialog modules corresponding to the structured components of traditional dialog systems and then incorporate these modules in a higher-level generative model. Structured Fusion Networks obtain strong results on the MultiWOZ dataset, both with and without reinforcement learning. Structured Fusion Networks are shown to have several valuable properties, including better domain generalizability, improved performance in reduced data scenarios and robustness to divergence during reinforcement learning.

MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems

“…However, the recently proposed multi-domain taskoriented dialogue datasets Eric et al, 2019) bring new challenges for multi-domain dialogue state tracking and response generation. Several follow up works ; Budzianowski and Vulić, 2019; Mehri et al, 2019;Madotto et al, 2020b) improved on the initial baselines with various methodologies. proposed the domain aware multi-decoder network and augmented the system act labels by leveraging the user act annotation, achieving the SOTA results in MultiWoz.…”

“…For the end-to-end dialogue modeling task, there are three automatic metrics to evaluate the response quality: 1) Inform rate: if the system provides a correct entity, 2) Success rate: if the system provides the correct entity and answers all the requested information, 3) BLEU (Papineni et al, 2002) for measuring the fluency of the generated response. Following previous work (Mehri et al, 2019), we also report the combined score, i.e., Combined = (Inform + Success)×0.5 + BLEU, as an overall quality measure. Joint goal accuracy (Joint Acc.)…”

“…SFN + RL: a seq2seq network comprised of several pre-trained dialogue modules that are connected through hidden states. Reinforcement fine tuning is used additionally to train the model (Mehri et al, 2019). MD-Sequicity: an extension of the Sequicity (Lei et al, 2018) framework for multi-domain task-oriented dialogue by .…”

A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning

“…End-to-end task-oriented dialog systems. Our model belongs to the family of E2E task-oriented dialog models (Wen et al, 2017a,b;Li et al, 2017;Mehri et al, 2019;Peng et al, 2020;Hosseini-Asl et al, 2020). We borrow some elements from the Sequicity ) model, such as representing the belief state as a natural language sequence (a text span), and using copy-augmented Seq2Seq learning (Gu et al, 2016).…”

“…The TSCP , SEDST , FSDM (Shu et al, 2019), MOSS (Liang et al, 2020) and DAMD are based on the copy-augmented Seq2Seq learning framework proposed by . LIDM (Wen et al, 2017a), SFN (Mehri et al, 2019) and UniConv (Le et al, 2020a) are modular designed, connected through neural states and trained end-to-end. SimpleTOD (Hosseini-Asl et al, 2020) and SOLOLIST (Peng et al, 2020) are two recent models, which both use a single autoregressive language model, initialized from GPT-2, to build the entire system.…”