Data Augmentation for Neural Online Chats Response Selection

Wen, Du; Black, Alan W.

doi:10.18653/v1/w18-5708

Cited by 13 publications

(11 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such models are typically evaluated using Recall@k, a typical metric in information retrieval literature. This measures how often the correct response is identified as one of the top k ranked responses (Lowe et al, 2015;Inaba and Takahashi, 2016;Yu et al, 2016;Al-Rfou et al, 2016;Henderson et al, 2017;Lowe et al, 2017;Wu et al, 2017;Chaudhuri et al, 2018;Du and Black, 2018;Kumar et al, 2018;Zhou et al, 2018;Gunasekara et al, 2019;Tao et al, 2019). Models trained to select responses can be used to drive dialogue systems, question-answering systems, and response suggestion systems.…”

Section: Response Selection Taskmentioning

confidence: 99%

A Repository of Conversational Datasets

Henderson¹,

Budzianowski²,

Casanueva³

et al. 2019

Proceedings of the First Workshop on NLP for Conversational AI

View full text Add to dashboard Cite

Progress in Machine Learning is often driven by the availability of large datasets, and consistent evaluation metrics for comparing modeling approaches. To this end, we present a repository of conversational datasets consisting of hundreds of millions of examples, and a standardised evaluation procedure for conversational response selection models using 1-of-100 accuracy. The repository contains scripts that allow researchers to reproduce the standard datasets, or to adapt the pre-processing and data filtering steps to their needs. We introduce and evaluate several competitive baselines for conversational response selection, whose implementations are shared in the repository, as well as a neural encoder model that is trained on the entire training set.

show abstract

Section: Response Selection Taskmentioning

confidence: 99%

A Repository of Conversational Datasets

Henderson¹,

Budzianowski²,

Casanueva³

et al. 2019

Proceedings of the First Workshop on NLP for Conversational AI

View full text Add to dashboard Cite

show abstract

“…Response selection is also directly applicable to retrieval-based dialog systems, a popular and elegant approach to framing dialog (Wu et al, 2017;Weston et al, 2018;Mazaré et al, 2018;Gunasekara et al, 2019;Henderson et al, 2019b). 1 Response Selection is a task of selecting the most appropriate response given the dialog history (Wang et al, 2013;Al-Rfou et al, 2016;Du and Black, 2018;Chaudhuri et al, 2018). This task is central to retrieval-based dialog systems, which typically encode the context and a large collection of responses in a joint semantic space, and then retrieve the most relevant response by matching the query representation against the encodings of each candidate response.…”

Section: Introductionmentioning

confidence: 99%

ConveRT: Efficient and Accurate Conversational Representations from Transformers

Henderson¹,

Casanueva²,

Mrkšić³

et al. 2020

Findings of the Association for Computational Linguistics: EMNLP 2020

116

105

View full text Add to dashboard Cite

General-purpose pretrained sentence encoders such as BERT are not ideal for real-world conversational AI applications; they are computationally heavy, slow, and expensive to train. We propose ConveRT (Conversational Representations from Transformers), a pretraining framework for conversational tasks satisfying all the following requirements: it is effective, affordable, and quick to train. We pretrain using a retrieval-based response selection task, effectively leveraging quantization and subword-level parameterization in the dual encoder to build a lightweight memoryand energy-efficient model. We show that Con-veRT achieves state-of-the-art performance across widely established response selection tasks. We also demonstrate that the use of extended dialog history as context yields further performance gains. Finally, we show that pretrained representations from the proposed encoder can be transferred to the intent classification task, yielding strong results across three diverse data sets. ConveRT trains substantially faster than standard sentence encoders or previous state-of-the-art dual encoders. With its reduced size and superior performance, we believe this model promises wider portability and scalability for Conversational AI applications.

show abstract

“…There exists a scarcity of the data required to train a dialog system for most tasks. Various methods have been proposed to tackle this issue including paraphrase techniques to generate artificial training data (Kumar et al, 2021;Du and Black, 2018), generating annotations including intent-slots and dialog acts (Yoo et al, 2019(Yoo et al, , 2020a or even injecting noise to improve robustness in dialog act prediction for ASR data (Wang et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions

Mohapatra¹,

Pandey²,

Contractor³

et al. 2021

Findings of the Association for Computational Linguistics: EMNLP 2021

View full text Add to dashboard Cite

Popular dialog data sets such as MultiWOZ (Budzianowski et al., 2018) are created by providing crowd workers an instruction, expressed in natural language, that describes the task to be accomplished. Crowd workers play the role of a user and an agent to generate dialogs to accomplish tasks involving booking restaurant tables, calling a taxi etc. In this paper, we present a data creation strategy that uses the pre-trained language model, GPT2 (Radford et al., 2018), to simulate the interaction between crowd workers by creating a user bot and an agent bot. We train the simulators using a smaller percentage of actual crowd-generated conversations and their corresponding instructions. We demonstrate that by using the simulated data, we achieve significant improvements in low-resource settings on two publicly available datasets -MultiWOZ dataset (Budzianowski et al., 2018) and the Persona chat dataset (Zhang et al., 2018a).

show abstract

Data Augmentation for Neural Online Chats Response Selection

Cited by 13 publications

References 22 publications

A Repository of Conversational Datasets

A Repository of Conversational Datasets

ConveRT: Efficient and Accurate Conversational Representations from Transformers

Simulated Chats for Building Dialog Systems: Learning to Generate Conversations from Instructions

Contact Info

Product

Resources

About