Neural Machine Translation For Paraphrase Generation

Sokolov, Alex; Filimonov, Denis

doi:10.48550/arxiv.2006.14223

Cited by 8 publications

(10 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Kazemnejad et al (2020) proposed a retrievalbased approach to retrieve paraphrase from a large corpus. ; Sokolov and Filimonov (2020) casted paraphrase generation as the task of machine translation. ; extended the idea of bilingual pivoting for paraphrase generation where the input sentence is first translated into a foreign language, and then translated back as the paraphrase.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ConRPG: Paraphrase Generation using Contexts as Regularizer

Meng¹,

et al. 2021

Preprint

View full text Add to dashboard Cite

A long-standing issue with paraphrase generation is how to obtain reliable supervision signals. In this paper, we propose an unsupervised paradigm for paraphrase generation based on the assumption that the probabilities of generating two sentences with the same meaning given the same context should be the same. Inspired by this fundamental idea, we propose a pipelined system which consists of paraphrase candidate generation based on contextual language models, candidate filtering using scoring functions, and paraphrase model training based on the selected candidates.The proposed paradigm offers merits over existing paraphrase generation methods: (1) using the context regularizer on meanings, the model is able to generate massive amounts of high-quality paraphrase pairs; and (2) using human-interpretable scoring functions to select paraphrase pairs from candidates, the proposed framework provides a channel for developers to intervene with the data generation process, leading to a more controllable model. Experimental results across different tasks and datasets demonstrate that the effectiveness of the proposed model in both supervised and unsupervised setups. 1

show abstract

Section: Related Workmentioning

confidence: 99%

“…; extended the idea of bilingual pivoting for paraphrase generation where the input sentence is first translated into a foreign language, and then translated back as the paraphrase. Sokolov and Filimonov (2020) trained a MT model using multilingual parallel data and then finetuned the model using parallel paraphrase data.…”

Section: Related Workmentioning

confidence: 99%

ConRPG: Paraphrase Generation using Contexts as Regularizer

Meng¹,

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…; Qian et al (2019) employed distinct semantic style embeddings to generate diverse paraphrases, and Iyyer et al (2018); ; Chen et al (2019); Goyal and Durrett (2020) proposed to use different syntactic structure templates. A line of work Sokolov and Filimonov, 2020) formalized paraphrase generation as machine translation. Unsupervised paraphrase generation is primarily based on reinforcement learning (RL) generative models (Ranzato et al, 2015;Li et al, 2016b).…”

Section: Related Workmentioning

confidence: 99%

“…Building a strong paraphrase generation system usually requires massive amounts of high-quality annotated paraphrase pairs, but existing labeled datasets (Lin et al, 2014;Fader et al, 2013;Lan et al, 2017) are either of small sizes or restricted in narrow domains. To avoid such a heavy reliance on labeled datasets, recent works have explored unsupervised methods (Li et al, 2018b;Fu et al, 2019;Siddique et al, 2020) to generate paraphrase without annotated training data, among which the backtranslation based model is an archetype Sokolov and Filimonov, 2020). It borrows the idea of back-translation (BT) in machine translation (Sennrich et al, 2016) where the model first translates a sentence s 1 into another sentence s 2 in a different language (e.g., En→Fr), and then translates s 2 back to s 1 .…”

Section: Introductionmentioning

confidence: 99%

Paraphrase Generation as Unsupervised Machine Translation

Fan¹,

Tian²,

Meng³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we propose a new paradigm for paraphrase generation by treating the task as unsupervised machine translation (UMT) based on the assumption that there must be pairs of sentences expressing the same meaning in a large-scale unlabeled monolingual corpus. The proposed paradigm first splits a large unlabeled corpus into multiple clusters, and trains multiple UMT models using pairs of these clusters. Then based on the paraphrase pairs produced by these UMT models, a unified surrogate model can be trained to serve as the final SEQ2SEQ model to generate paraphrases, which can be directly used for test in the unsupervised setup, or be finetuned on labeled datasets in the supervised setup. The proposed method offers merits over machinetranslation-based paraphrase generation methods, as it avoids reliance on bilingual sentence pairs. It also allows human intervene with the model so that more diverse paraphrases can be generated using different filtering criteria. Extensive experiments on existing paraphrase dataset for both the supervised and unsupervised setups demonstrate the effectiveness the proposed paradigm.

show abstract

“…Malandrakis et al (2019) introduce a similar notion of paraphrasing and apply variational autoencoders to control the quality of paraphrases. Sokolov and Filimonov (2020) tackle a similar problem of paraphrasing utterances with entity types, but implement the slot copy mechanism via pre-processing and post-processing. In addition, Liu et al (2013) apply paraphrases to improve natural understanding in an NLU system, both for augmenting rules and for enhancing features.…”

Section: Related Workmentioning

confidence: 99%

Delexicalized Paraphrase Generation

Arkoudas

Hamza

2020

Preprint

View full text Add to dashboard Cite

We present a neural model for paraphrasing and train it to generate delexicalized sentences. We achieve this by creating training data in which each input is paired with a number of reference paraphrases. These sets of reference paraphrases represent a weak type of semantic equivalence based on annotated slots and intents. To understand semantics from different types of slots, other than anonymizing slots, we apply convolutional neural networks (CNN) prior to pooling on slot values and use pointers to locate slots in the output. We show empirically that the generated paraphrases are of high quality, leading to an additional 1.29% exact match on live utterances. We also show that natural language understanding (NLU) tasks, such as intent classification and named entity recognition, can benefit from data augmentation using automatically generated paraphrases.

show abstract

Neural Machine Translation For Paraphrase Generation

Cited by 8 publications

References 0 publications

ConRPG: Paraphrase Generation using Contexts as Regularizer

ConRPG: Paraphrase Generation using Contexts as Regularizer

Paraphrase Generation as Unsupervised Machine Translation

Delexicalized Paraphrase Generation

Contact Info

Product

Resources

About