The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems

Abstract

This paper introduces the Ubuntu Dialogue Corpus, a dataset containing almost 1 million multi-turn dialogues, with a total of over 7 million utterances and 100 million words. This provides a unique resource for research into building dialogue managers based on neural language models that can make use of large amounts of unlabeled data. The dataset has both the multi-turn property of conversations in the Dialog State Tracking Challenge datasets, and the unstructured nature of interactions from microblog services such as Twitter. We also describe two neural learning architectures suitable for analyzing this dataset, and provide benchmark performance on the task of selecting the best next response.

On the Evaluation of Dialogue Systems with Next Utterance Classification

“…The human results are separated into AMT non-experts, consisting of paid respondents who have 'Beginner' or no knowledge of Ubuntu terminology; AMT experts, who claimed to have 'Intermediate' or 'Advanced' knowledge of Ubuntu; and Lab experts, who are the non-paid respondents with Ubuntu experience and university-level computer science training. We also presents results on the same task for a state-of-the-art artificial neural (Lowe et al, 2015a) Table 2: Average results on each corpus. 'Number of Users' indicates the number of respondents for each category.…”

“…the actual response of the conversation) that are found in the top k responses with the highest rankings according to the model. This task has gained some popularity recently for evaluating dialogue systems (Lowe et al, 2015a;Kadlec et al, 2015).…”

“…In this paper, we consider to what extent NUC is achievable by humans, whether human performance varies according to expertise, and whether there is room for machine performance to improve (or has reached human performance already) and we should move to more complex conversational tasks. We performed a user study on three different datasets: the SubTle Corpus of movie dialogues (Banchs, 2012), the Twitter Corpus (Ritter et al, 2010), and the Ubuntu Dialogue Corpus (Lowe et al, 2015a). Since conversations in the Ubuntu Dialogue Corpus are highly technical, we recruit 'expert' humans who are adept with the Ubuntu terminology, whom we compare with a state-of-the-art machine learning agent on all datasets.…”

Chinese Short Text Summary Generation Model Combining Global and Local Information

“…Finally, after a fully connected layer and cross-entropy loss, a two-category model of matching degree between text and abstract is obtained. This model is based on the simple improvement of the Dual-LSTM model [12] in the Q & A retrieval field, which is better than the direct application of the text digest to match degree, both accuracy and recall. (3) First, the two-category matching model for text and abstract is used to predict the score of the first part of the LSCTS data set and a positive sample with a score of more than 0.95 and a negative sample of less than 0.1 are taken out and added to the training set again.…”

Working of Acontext-Aware Conversational Entity

“…After this, it finds a list of possible synonyms for the word according to its POS. At the end, it checks which word exists in the vocabulary, and according to that, it alters the sentence [15]. For all those words whose synonym does not exist or whose synonyms are not in the vocabulary are left unchanged.…”