Quality Controlled Paraphrase Generation

Elron, Bandel,; Aharonov, Ranit; Shmueli-Scheuer, Michal; Shnayderman, Ilya; Slonim, Noam; Ein-Dor, Liat

doi:10.18653/v1/2022.acl-long.45

Cited by 14 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(Liu and Soh, 2022a). This approach is similar to Bandel et al (2022); 's proposed metrics for paraphrase quality (e.g., semantic similarity, expression diversity).…”

Section: Related Workmentioning

confidence: 98%

“…Recent work requires texts to satisfy certain stylistic, semantic, or structural requirements, such as using formal language or expressing thoughts using a particular template (Iyyer et al, 2018;Shen et al, 2017). In paraphrase generation, methods require texts to meet certain quality criteria, such as semantic preservation and lexical diversity (Bandel et al, 2022;Yang et al, 2022) or require syntactic criteria, such as word ordering Goyal and Durrett, 2020;. The development of the Multi-Topic Paraphrase in Twitter (MultiPIT) corpus addresses quality issues in existing paraphrase datasets and facilitates the acquisition and generation of high-quality paraphrases (Dou et al, 2022).…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

Wahle

Ruas

Meuschke

et al. 2023

Preprint

View full text Add to dashboard Cite

Neural language models such as BERT allow for human-like text paraphrasing. This ability threatens academic integrity, as it aggravates identifying machine-obfuscated plagiarism. We make two contributions to foster the research on detecting these novel machine-paraphrases. First, we provide the first large-scale dataset of documents paraphrased using the Transformer-based models BERT, RoBERTa, and Longformer. The dataset includes paragraphs from scientific papers on arXiv, theses, and Wikipedia articles and their paraphrased counterparts (1.5M paragraphs in total). We show the paraphrased text maintains the semantics of the original source. Second, we benchmark how well neural classification models can distinguish the original and paraphrased text. The dataset and source code of our study are publicly available.

show abstract

“…(Liu and Soh, 2022a). This approach is similar to Bandel et al (2022); 's proposed metrics for paraphrase quality (e.g., semantic similarity, expression diversity).…”

Section: Related Workmentioning

confidence: 98%

Section: Related Workmentioning

confidence: 99%

Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

Wahle

Ruas

Meuschke

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Algorithm 2 summarises the candidate generation procedure, which takes any paraphrase model, a list of model-specific generation parameters and, optionally, a list of filters as an input (line 1). These parameters are temperature and number of beams for Pegasus, or a grid of lexical, semantic and syntactic distances for the Quality Controlled Paraphrase Generation (QCPG) (Bandel et al, 2022) model presented in Appendix E. The model generates one or more paraphrases, which are filtered before returning (lines 3-8). We describe the filtering process next.…”

Section: B1 Candidate Generationmentioning

confidence: 99%

“…In Table 7 we show that simply re-ranking the Pegasus output space with BLEURT improves SGD performance comparably with backtranslation (rows 2&4) and the robustness and generalisation improvements are maintained. Bandel et al (2022) exploit high quality examples in paraphrase corpora by conditioning the model with a string quality parameters string outlining target semantic, syntactic and lexical distances of the generated paraphrase during finetuning. At inference one must specify these parameters to obtain diverse yet high quality paraphrases.…”

Section: Filter Namementioning

confidence: 99%

More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking

Coca,

Tseng,

Lin

et al. 2023

Findings of the Association for Computational Linguistics: EACL 2023

View full text Add to dashboard Cite

The schema-guided paradigm overcomes scalability issues inherent in building task-oriented dialogue (TOD) agents with static ontologies. Instead of operating on dialogue context alone, agents have access to hierarchical schemas containing task-relevant natural language descriptions. Fine-tuned language models excel at schema-guided dialogue state tracking (DST) but are sensitive to the writing style of the schemas. We explore methods for improving the robustness of DST models. We propose a framework 1 for generating synthetic schemas which uses tree-based ranking to jointly optimise lexical diversity and semantic faithfulness. The generalisation of strong baselines is improved when augmenting their training data with prompts generated by our framework, as demonstrated by marked improvements in average joint goal accuracy (JGA) and schema sensitivity (SS) on the SGD-X benchmark.

show abstract

“…4) Since our focus was on evaluating machine translation, we naturally chose translation for augmenting the data. However, other data augmentation techniques could seamlessly integrate into DATScore, such as using a text paraphrasing model (Bandel et al, 2022).…”

Section: Limitationsmentioning

confidence: 99%

FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metrics for Automatic Text Generation

Eddine¹,

Shang

Tixier³

et al. 2022

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

View full text Add to dashboard Cite

Fast and reliable evaluation metrics are key to R&D progress. While traditional natural language generation metrics are fast, they are not very reliable. Conversely, new metrics based on large pretrained language models are much more reliable, but require significant computational resources. In this paper, we propose Fru-galScore, an approach to learn a fixed, low cost version of any expensive NLG metric, while retaining most of its original performance. Experiments with BERTScore and MoverScore on summarization and translation show that Fru-galScore is on par with the original metrics (and sometimes better), while having several orders of magnitude less parameters and running several times faster. On average over all learned metrics, tasks, and variants, FrugalScore retains 96.8% of the performance, runs 24 times faster, and has 35 times less parameters than the original metrics. We make our trained metrics publicly available 1 and easily accessible via Hugging Face, to benefit the entire NLP community and in particular researchers and practitioners with limited resources.

show abstract

Quality Controlled Paraphrase Generation

Cited by 14 publications

References 0 publications

Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

More Robust Schema-Guided Dialogue State Tracking via Tree-Based Paraphrase Ranking

FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metrics for Automatic Text Generation

Contact Info

Product

Resources

About