Sentence alignment using local and global information

Zamani, Hamed; Faili, Heshaam; Shakery, Azadeh

doi:10.1016/j.csl.2016.03.002

Cited by 7 publications

(5 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For building aligned paired sentences, first of all, the aligned English-Persian documents were extracted from Wikipedia. In the second step, in order to extract aligned sentences, a Maximum Entropy binary classifier has been trained in order to evaluate the local similarity between sentence pairs in Persian and English aligned documents [23]. MaxEnt classifiers are log-linear models that try to capture contextual information.…”

Section: -1-1-extracting Parallel Sentencesmentioning

confidence: 99%

“…Barrón-Cedeño, Paramita, Clough, and Rosso have investigated cross-language similarities by incorporating various features such as character n-grams, cognateness, word count ratio, and an approach based on out-links in Wikipedia pages [24]. In this research, a collection of 12 features in four categories were exploited to train the maximum entropy classifier [23]. The four categories are as follows:…”

Section: -1-1-extracting Parallel Sentencesmentioning

confidence: 99%

“…The features for the construction of a parallel corpus have been thoroughly described in [23]. In order to train the MaxEnt classifier, a non-parallel corpus was also constructed to compensate the bias toward the parallel sentence.…”

Section: Miscellaneous Featuresmentioning

confidence: 99%

See 2 more Smart Citations

A Corpus for Evaluation of Cross Language Text Re-use Detection Systems

Mohtaj¹,

Asghari²

2022

jist

View full text Add to dashboard Cite

In recent years, the availability of documents through the Internet along with automatic translation systems have increased plagiarism, especially across languages. Cross-lingual plagiarism occurs when the source or original text is in one language and the plagiarized or re-used text is in another language. Various methods for automatic text re-use detection across languages have been developed whose objective is to assist human experts in analyzing documents for plagiarism cases. For evaluating the performance of these systems and algorithms, standard evaluation resources are needed. To construct cross lingual plagiarism detection corpora, the majority of earlier studies have paid attention to English and other European language pairs, and have less focused on low resource languages. In this paper, we investigate a method for constructing an English-Persian cross-language plagiarism detection corpus based on parallel bilingual sentences that artificially generate passages with various degrees of paraphrasing. The plagiarized passages are inserted into topically related English and Persian Wikipedia articles in order to have more realistic text documents. The proposed approach can be applied to other less-resourced languages. In order to evaluate the compiled corpus, both intrinsic and extrinsic evaluation methods were employed. So, the compiled corpus can be suitably included into an evaluation framework for assessing cross-language plagiarism detection systems. Our proposed corpus is free and publicly available for research purposes.

show abstract

Section: -1-1-extracting Parallel Sentencesmentioning

confidence: 99%

Section: -1-1-extracting Parallel Sentencesmentioning

confidence: 99%

See 1 more Smart Citation

A Corpus for Evaluation of Cross Language Text Re-use Detection Systems

Mohtaj¹,

Asghari²

2022

jist

View full text Add to dashboard Cite

show abstract

“…This is somehow similar to our approach based on DTW, although we directly operate at the sentence level. To address the task of aligning documents in which sentences do not appear in the same order, Zamani et al (2016) present an approach based on Integer Linear Programming, while Quan et al (2018) propose instead an approach based on the length of the sentences to create one-to-one matches. Other examples of alignement between parallel corpora can be found in (Santos, 2011).…”

Section: Related Workmentioning

confidence: 99%

“…None of the reviewed approaches is directly applicable to our setting. The works of Das andPetrov (2011), Moore (2002), Quan et al (2018), andZamani et al (2016) are all based on symmetric corpora, while imposing a 1-1 alignment is not possible in our domain. Bamman et al (2010), Bentivogli and Pianta (2005), Eger et al (2018), Fossum and Abney (2005), and Yarowsky et al (2001) all address word-level alignment instead of sentence level-alignment.…”

Section: Related Workmentioning

confidence: 99%

Cross-lingual Annotation Projection in Legal Texts

Galassi

Drazewski²,

Lippi

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

View full text Add to dashboard Cite

We study annotation projection in text classification problems where source documents are published in multiple languages and may not be an exact translation of one another. In particular, we focus on the detection of unfair clauses in privacy policies and terms of service. We present the first English-German parallel asymmetric corpus for the task at hand. We study and compare several language-agnostic sentence-level projection methods. Our results indicate that a combination of word embeddings and dynamic time warping performs best.

show abstract