Improving Limited Labeled Dialogue State Tracking with Self-Supervision

Abstract

Existing dialogue state tracking (DST) models require plenty of labeled data. However, collecting high-quality labels is costly, especially when the number of domains increases. In this paper, we address a practical DST problem that is rarely discussed, i.e., learning efficiently with limited labeled data. We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior. We encourage a DST model to have consistent latent distributions given a perturbed input, making it more robust to an unseen scenario. We also add an auxiliary utterance generation task, modeling a potential correlation between conversational behavior and dialogue states. The experimental results show that our proposed self-supervised signals can improve joint goal accuracy by 8.95% when only 1% labeled data is used on the MultiWOZ dataset. We can achieve an additional 1.76% improvement if some unlabeled data is jointly trained as semi-supervised learning. We analyze and visualize how our proposed self-supervised signals help the DST task and hope to stimulate future data-efficient DST research.

Controllable Abstractive Dialogue Summarization with Sketch Supervision

“…Liu et al, 2019 collect their dataset from the logs in the DiDi customer service center. It is restricted to taskoriented scenario, where one speaker is the user and the other is the customer agent, with limited topics and it is also connected to the goal of dialogue state tracking task (Wu et al, 2019a(Wu et al, , 2020b. Recently, Gliwa et al, 2019 introduce the SAMSum corpus, with 16k chat dialogues with manually annotated summaries.…”

Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Tracking

“…Few-Shot DST is a promising direction for reducing the need of human annotation while achiev-ing quasi-SOTA performance with a fraction of the training data. Different techniques have been proposed (Wu et al, 2019;Mi et al, 2021;Li et al, 2021b;Gao et al, 2020;Lin et al, 2021b,a;Campagna et al, 2020;Wu et al, 2020b;Su et al, 2021;Peng et al, 2020;. We briefly describe and compare DS2 with existing few-shot models in Section 4.5.…”

Bridging the Gap between Language Model and Reading Comprehension: Unsupervised MRC via Self-Supervision

“…Self-supervised Learning for NLP. Recent studies have verified the effectiveness of selfsupervised learning (Raina et al, 2007) for different NLP tasks by designing proper pretext tasks (Wang et al, 2019;Wu et al, 2019;Banerjee and Baral, 2020;Wu et al, 2020;Rücklé et al, 2020;Shi et al, 2020;Yamada et al, 2020;Xu et al, 2020;Guu et al, 2020). For extractive summarization, Wang et al (2019) design pretext tasks including masking, replacing, and switching sentences in passages to learn contextualized representations.…”

“…For extractive summarization, Wang et al (2019) design pretext tasks including masking, replacing, and switching sentences in passages to learn contextualized representations. On dialogue generation, through the inconsistent order detection task (Wu et al, 2019) or utterance prediction and restoration tasks (Wu et al, 2020;Xu et al, 2020), models can learn to capture relationships between utterances in dialogue flows. On question answering, Banerjee and Baral (2020) propose the Knowledge Triplet Learning task to learn multiple-choice QA, and Rücklé et al (2020) use self-supervision for unsupervised transfer of answer matching ability among domains.…”