Learning to Converse with Noisy Data: Generation with Calibration

Shang, Mingyue; Fu, Zhenxin; Peng, Nanyun; Feng, Yansong; Zhao, Dongyan; Yan, Rui

doi:10.24963/ijcai.2018/603

Cited by 32 publications

(28 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to the model approach, models are trained while handling noise at the same time. For example, Shang et al (2018) proposed a method with a calibration framework and demonstrated its effectiveness on a Chinese corpus. According to the data approach, training data are pre-processed with the aim of improving their quality before training models.…”

Section: Introductionmentioning

confidence: 99%

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

Akama¹,

Yokoi

Suzuki

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

Large-scale dialogue datasets have recently become available for training neural dialogue agents. However, these datasets have been reported to contain a non-negligible number of unacceptable utterance pairs. In this paper, we propose a method for scoring the quality of utterance pairs in terms of their connectivity and relatedness. The proposed scoring method is designed based on findings widely shared in the dialogue and linguistics research communities. We demonstrate that it has a relatively good correlation with the human judgment of dialogue quality. Furthermore, the method is applied to filter out potentially unacceptable utterance pairs from a large-scale noisy dialogue corpus to ensure its quality. We experimentally confirm that training data filtered by the proposed method improves the quality of neural dialogue agents in response generation. 1

show abstract

Section: Introductionmentioning

confidence: 99%

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

Akama¹,

Yokoi

Suzuki

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

View full text Add to dashboard Cite

show abstract

“…Many efficient approaches have been proposed for developing intelligent dialogue systems (He et al, 2017;Shang et al, 2018;Tian et al, 2019;Cai et al, 2019). For multi-turn dialogue systems, Serban et al (2016) and Xing et al (2018) adopt hierarchical neural networks to model context.…”

Section: Multi-turn Dialogue Systemsmentioning

confidence: 99%

Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration

Pan¹,

Bai²,

Wang³

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

In multi-turn dialogue, utterances do not always take the full form of sentences. These incomplete utterances will greatly reduce the performance of open-domain dialogue systems. Restoring more incomplete utterances from context could potentially help the systems generate more relevant responses. To facilitate the study of incomplete utterance restoration for open-domain dialogue systems, a large-scale multi-turn dataset Restoration-200K 1 is collected and manually labeled with the explicit relation between an utterance and its context. We also propose a "pick-andcombine" model to restore the incomplete utterance from its context. Experimental results demonstrate that the annotated dataset and the proposed approach significantly boost the response quality of both single-turn and multiturn dialogue systems.

show abstract

“…Our work is inspired by the work of using new learning strategies to distinguish the noise in training data [7,10,15]. Shang et al [10] and Lison et al [7] utilized instance weighting strategy in open domain dialog systems via simple methods. Wu et al [15] altered the negative sampling strategy and utilized a sequence-to-sequence model to distinguish false negative samples.…”

Section: Related Workmentioning

confidence: 99%

Improving Multi-turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting

Zhou

Zhao

Zhu

et al. 2020

Advances in Knowledge Discovery and Data Mining

View full text Add to dashboard Cite

Open-domain retrieval-based dialogue systems require a considerable amount of training data to learn their parameters. However, in practice, the negative samples of training data are usually selected from an unannotated conversation data set at random. The generated training data is likely to contain noise and affect the performance of the response selection models. To address this difficulty, we consider utilizing the underlying correlation in the data resource itself to derive different kinds of supervision signals and reduce the influence of noisy data. More specially, we consider a main-complementary task pair. The main task (i.e., our focus) selects the correct response given the last utterance and context, and the complementary task selects the last utterance given the response and context. The key point is that the output of the complementary task is used to set instance weights for the main task. We conduct extensive experiments in two public datasets and obtain significant improvement in both datasets. We also investigate the variant of our approach in multiple aspects, and the results have verified the effectiveness of our approach.

show abstract

Learning to Converse with Noisy Data: Generation with Calibration

Cited by 32 publications

References 5 publications

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

Filtering Noisy Dialogue Corpora by Connectivity and Content Relatedness

Improving Open-Domain Dialogue Systems via Multi-Turn Incomplete Utterance Restoration

Improving Multi-turn Response Selection Models with Complementary Last-Utterance Selection by Instance Weighting

Contact Info

Product

Resources

About