A Deep Generative Framework for Paraphrase Generation

Abstract

Paraphrase generation is an important problem in NLP, especially in question answering, information retrieval, information extraction, conversation systems, to name a few. In this paper, we address the problem of generating paraphrases automatically. Our proposed method is based on a combination of deep generative models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases, given an input sentence. Traditional VAEs when combined with recurrent neural networks can generate free text but they are not suitable for paraphrase generation for a given sentence. We address this problem by conditioning the both, encoder and decoder sides of VAE, on the original sentence, so that it can generate the given sentence's paraphrases. Unlike most existing models, our model is simple, modular and can generate multiple paraphrases, for a given sentence. Quantitative evaluation of the proposed method on a benchmark paraphrase dataset demonstrates its efficacy, and its performance improvement over the state-of-the-art methods by a significant margin, whereas qualitative human evaluation indicate that the generated paraphrases are well-formed, grammatically correct, and are relevant to the input sentence. Furthermore, we evaluate our method on a newly released question paraphrase dataset, and establish a new baseline for future research.

Syntactically Robust Training on Partially-Observed Data for Open Information Extraction

“…Paraphrase Generation has proven to be useful for adversarial training and data augmentation (Zhou and Bhat, 2021). Early methods adopt hand-crafted rules (McKeown, 1983), synonym substitution (Bolshakov and Gelbukh, 2004), machine translation (Quirk et al, 2004), and deep learning (Gupta et al, 2018; to improve the quality of generated sentences. To acquire syntactic diverse samples, recent studies involve reinforcement learning (Qian et al, 2019) or syntactic constrains (Iyyer et al, 2018;Goyal and Durrett, 2020;Sun et al, 2021) into the models.…”

Deep Latent Variable Models for Semi-supervised Paraphrase Generation

“…Paraphrases express the surface forms of the underlying semantic content [6] and capture the essence of language diversity [46]. Early work on automatic generation of paraphrase are generally rule-based [7,8], but the recent trend brings to fore neural network solutions [47,19,10,12,13,20,6]. Current research for paraphrasing mainly focuses on supervised methods, which require the availability of a large number of source and target pairs.…”

Paraphrase Generation Model Integrating Transformer Architecture, Part-of-Speech Features, and Pointer Generator Network

“…Prakash et al [9] and Gupta et al [10] have proposed applying the Seq2Seq model, which is based on Long Short-2 Term Memory (LSTM), for paraphrasing. Prakash et al noted that applying the residual network architecture [11] to the stacked LSTM network architecture [9] was beneficial for training.…”