Self-Supervised Dialogue Learning

Wu, Jiawei; Wang, Xin; Wang, William Yang

doi:10.18653/v1/p19-1375

Cited by 61 publications

(39 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, our model can provide high-quality responses at a low cost. Before us, there have been a few studies on learning a primary task with auxiliary ones (Rei and Yannakoudakis, 2017;Yu and Jiang, 2016;Ding et al, 2017;Trinh et al, 2018;Mehri et al, 2019;Wu et al, 2019). The work is unique in that through extensive empirical studies, we verified that a simple structure learned with auxiliary tasks can work as well as deep architectures in dialogue generation.…”

Section: Related Workmentioning

confidence: 66%

“…(2) VHRED 3 : an extension of HRED that factorizes response generation with latent variables (Serban et al, 2017); (3) HRAN 4 : hierarchical encoder-decoder equipped with a hierarchical attention mechanism (Xing et al, 2018); (4) ReCoSa 5 : a hierarchical transformer-based model that exhibits state-of-the-art performance on benchmarks ; and (5) SSN: a very recent study on enhancing dialogue generation learning with self-supervision signals extracted from utterance order (Wu et al, 2019).…”

Section: Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks

Zhao¹,

Xu²,

Wu³

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

We study multi-turn response generation for open-domain dialogues. The existing state-ofthe-art addresses the problem with deep neural architectures. While these models improved response quality, their complexity also hinders the application of the models in real systems. In this work, we pursue a model that has a simple structure yet can effectively leverage conversation contexts for response generation. To this end, we propose four auxiliary tasks including word order recovery, utterance order recovery, masked word recovery, and masked utterance recovery, and optimize the objectives of these tasks together with maximizing the likelihood of generation. By this means, the auxiliary tasks that relate to context understanding can guide the learning of the generation model to achieve a better local optimum. Empirical studies with three benchmarks indicate that our model can significantly outperform state-of-the-art generation models in terms of response quality on both automatic evaluation and human judgment, and at the same time enjoys a much faster decoding process.

show abstract

Section: Related Workmentioning

confidence: 66%

Section: Baselinesmentioning

confidence: 99%

Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks

Zhao¹,

Xu²,

Wu³

2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…In contrast to them, we take a cross-domain QA framework that trains the QA model with source domain data (virtual-world data) and generates answers with target domain data (real-world data.) Another setting of natural answer generation is the setting of multiturn QA (i.e., dialogue) [60], [61], which generates answer responses based on a given context and a history, including past questions and answers. Our story-based QA can be viewed as a special case of single-turn dialogue, which does not use any past questions and answers.…”

Section: Natural Answer Generationmentioning

confidence: 99%

Sim2RealQA: Using Life Simulation to Solve Question Answering Real-World Events

2021

View full text Add to dashboard Cite

As smart speakers continue to proliferate, question answering (QA) by smart devices is being woven into our daily lives. This study assumes question answering related to daily life events detected by context recognition systems, such as activity recognition and indoor positioning systems, e.g., answering questions like "Did my grandma eat dinner?" and "How many times did my grandpa go to the toilet?" These questions can effectively support human memory-aids, locate lost items, and monitor human activities. However, training a question-answering model requires large amounts of labeled training data (i.e., questions, answers, and the time-series of real-world event triplets) collected in a target environment. In this paper, we propose a novel simulation to real QA (Sim2RealQA) framework that completely trains a QA model with QA datasets produced in a life simulator and use it for solving real-word QA problems without answer labels. Our proposed QA model can learn a general reasoning process for QA that is independent of environments and deal with diverse types of questions specific to question answering in real-world environments, e.g., counting the number of occurrences of a real-world event and enumerating the names of those who are performing an activity together. Experiments show that using life simulations is a promising approach for solving real-world QA problems when no real-world answer labels are available.INDEX TERMS Lifelog, ubiquitous computing, daily-living activities, real-world question answering.

show abstract

“…Shi et al (2019) used variational RNN to extract latent dialogue structure and applied it to dialogue policy learning. Wu et al (2019b) introduced a self-supervised learning task, inconsistent order detection, to explicitly capture the flow of conversation in dialogues. Jin et al (2018) use unlabeled data to train probabilistic distributions over the vocabulary space as dialogue states for neural dialogue generation.…”

Section: Related Workmentioning

confidence: 99%

Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Wu¹,

Hoi

Xiong³

2020

Findings of the Association for Computational Linguistics: EMNLP 2020

View full text Add to dashboard Cite

Existing dialogue state tracking (DST) models require plenty of labeled data. However, collecting high-quality labels is costly, especially when the number of domains increases. In this paper, we address a practical DST problem that is rarely discussed, i.e., learning efficiently with limited labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. We encourage a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario. We also add an auxiliary utterance generation task, modeling a potential correlation between conversational behavior and dialogue states. The experimental results show that our proposed self-supervised signals can improve joint goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset. We can achieve an additional 1.76% improvement if some unlabeled data is jointly trained as semi-supervised learning. We analyze and visualize how our proposed self-supervised signals help the DST task and hope to stimulate future data-efficient DST research.

show abstract

Self-Supervised Dialogue Learning

Cited by 61 publications

References 36 publications

Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks

Learning a Simple and Effective Model for Multi-turn Response Generation with Auxiliary Tasks

Sim2RealQA: Using Life Simulation to Solve Question Answering Real-World Events

Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Contact Info

Product

Resources

About