Unsupervised construction of large paraphrase corpora

Abstract

We investigate unsupervised techniques for acquiring monolingual sentence-level paraphrases from a corpus of temporally and topically clustered news articles collected from thousands of web-based news sources. Two techniques are employed: (1) simple string edit distance, and (2) a heuristic strategy that pairs initial (presumably summary) sentences from different news stories in the same cluster. We evaluate both datasets using a word alignment algorithm and a metric borrowed from machine translation. Results show that edit distance data is cleaner and more easily-aligned than the heuristic data, with an overall alignment error rate (AER) of 11.58% on a similarly-extracted test set. On test data extracted by the heuristic strategy, however, performance of the two training sets is similar, with AERs of 13.2% and 14.7% respectively. Analysis of 100 pairs of sentences from each set reveals that the edit distance data lacks many of the complex lexical and syntactic alternations that characterize monolingual paraphrase. The summary sentences, while less readily alignable, retain more of the non-trivial alternations that are of greatest interest learning paraphrase relationships.

Comparable Evaluation of Contemporary Corpus-Based and Knowledge-Based Semantic Similarity Measures of Short Texts

“…For the purpose of evaluation, we used Microsoft Shared Paraphrase Corpus (MSPC, [1]). It consists of 5081 pairs of sentences graded with a binary 0 for semantically non-similar and binary 1 for semantically similar.…”

“…We tried to improve this method by adding a measure of a weight to each word, so the words with a bigger weight would factor more in the overall evaluation of sentence similarity. Next, we imposed a modifi cation of a text similarity scoring function, defi ned in [8], where the similarity between the input text segments 1 T and 2 T is determined by using the following scoring function:…”

Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics

“…Since our solution to IER establishes a relationship between entity definitions and the input text, the tasks of paraphrase recognition (Barzilay and Elhadad, 2003) (Dolan et al, 2004) and textual entailment recognition are related to our solution. However, these tasks are fundamentally different in two aspects: 1) Both paraphrase recognition and textual entailment recogni-229 tion are defined at the sentence level, whereas text phrases considered for IER can exist as a sentence fragment or span across multiple sentences, and 2) The objective of IER is to find whether a given text phrase has a mention of an entity-as opposed to determining whether two sentences are similar or entail one another.…”

Déjà Image-Captions: A Corpus of Expressive Descriptions in Repetition

“…Paraphrasing: Most previous studies in paraphrasing have focused exclusively on text, and the primary goal has been learning semantic equivalence of phrases that would be true out of context (e.g., (Barzilay and McKeown, 2001;Pang et al, 2003;Dolan et al, 2004;Ganitkevitch et al, 2013)), rather than targeting situated or pragmatic equivalence given a context. Emerging efforts began exploring paraphrases that are situated in video content (Chen and Dolan, 2011), news events (Zhang and Weld, 2013), and knowledge base (Berant and Liang, 2014).…”