Neural Paraphrase Identification of Questions with Noisy Pretraining

Tomar, Gaurav Singh; Duque, Thyago S.P.C.; Täckström, Oscar; Uszkoreit, Jakob; Das, Dipanjan

doi:10.18653/v1/w17-4121

Cited by 40 publications

(26 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They find that Shortcut-Stacked Sentence Encoder Model (SSE) [39] performs the best, giving testing accuracy of 87.8%. The previous best accuracy is 88.4% [13]. Our multi-cascaded model beats this result without any data augmentation with an accuracy of 89.4%.…”

Section: Quora Datasetsupporting

confidence: 52%

“…In contrast to [7], we avoid using ensemble model approach (which is computationally costly). Similarly, contrary to results in [13], our results are based on word features only and do not use computationally expensive character-based features.…”

Section: Quora Datasetmentioning

confidence: 88%

“…For instance, the AskUbuntu dataset [27] contains very few annotations, thus limiting the generalization performance of the model [13]. The ability to augment the data with additional sound annotations without requiring human intervention can improve the performance of deep models [4,13]. Such data augmentation has been shown to be fruitful for data analytics when only a piece of limited ordinal information about the pairwise distance between objects is provided [28,29,30].…”

Section: Related Workmentioning

confidence: 99%

“…Many deep learning-based schemes employ one or two Convolutional Neural Network (CNN) or Long Short Term Memory (LSTM) based models to learn features and make predictions on clean texts [7,12,13], while a recent model also incorporates linguistic features to detect paraphrases in both clean and noisy short texts [4]. For many NLP tasks involving short texts, it has been shown that developing wider models can yield significant gains [14].…”

Section: Introductionmentioning

confidence: 99%

“…crowd-sourcing) is costly [16]. Therefore, [4] and [13] add to the training set each labeled pair also in the reversed order. However, this simple data augmentation strategy can be extended in a systematic manner by relying upon set and graph theory.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

A multi-cascaded model with data augmentation for enhanced paraphrase detection in short texts

Shakeel

Karim

Khan

2020

Information Processing & Management

View full text Add to dashboard Cite

Paraphrase detection is an important task in text analytics with numerous applications such as plagiarism detection, duplicate question identification, and enhanced customer support helpdesks. Deep models have been proposed for representing and classifying paraphrases. These models, however, require large quantities of human-labeled data, which is expensive to obtain. In this work, we present a data augmentation strategy and a multi-cascaded model for improved paraphrase detection in short texts. Our data augmentation strategy considers the notions of paraphrases and non-paraphrases as binary relations over the set of texts. Subsequently, it uses graph theoretic concepts to efficiently generate additional paraphrase and non-paraphrase pairs in a sound manner. Our multi-cascaded model employs three supervised feature learners (cascades) based on CNN and LSTM networks with and without soft-attention. The learned features, together with hand-crafted linguistic features, are then forwarded to a discriminator network for final classification. Our model is both wide and deep and provides greater robustness across clean and noisy short texts. We evaluate our approach on three benchmark datasets and show that it produces a comparable or state-of-the-art performance on all three.• We present an efficient strategy for augmenting existing paraphrase and non-paraphrase annotations in a consistent manner. This strategy generates additional annotations and enhances the performance of the data-hungry deep learning models.• We develop a multi-cascaded learning model for robust paraphrase detection in both clean and noisy texts. This model incorporates multiple learned and linguistic features in a wide and deep discriminator network for paraphrase detection.• We address both clean and noisy texts in our presentation and show that the proposed model matches current best performances on benchmark datasets of both types.• We analyze the impact of various data augmentation steps and different components of the multicascaded model on paraphrase detection performance.

show abstract

Section: Quora Datasetsupporting

confidence: 52%

Section: Quora Datasetmentioning

confidence: 88%