Introduction

Skadiņa, Inguna; Gaizauskas, Robert; Vasiļjevs, Andrejs; Paramita, Monica Lestari

doi:10.1007/978-3-319-99004-0_1

Cited by 1 publication

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data driven NLP tasks depend a lot on the quality of the resources used, and the better the corpus is, the better knowledge one can extract from it. We thus conjecture that comparable corpora of higher quality 3 will yield better performance of applications relying on them, a fact that has actually been validated in several previous studies (Li and Gaussier 2010; Skadina et al 2010). Existing work mining comparable corpora mostly builds and uses comparable corpora according to humans’ simple intuitions.…”

Section: Introductionsupporting

confidence: 68%

“…This measure is however computationally infeasible if the corpora contain a large number of documents. Under the seventh European framework, 6 researchers involved in the project ACCURAT (Skadina et al 2010) have studied several measures and metrics for assessing corpus comparability and document parallelism of under-resourced languages. In addition to the above studies devoted to comparable corpora, researchers have recently developed several cross-lingual models of word embeddings (Hermann and Blunsom 2014; Luong, Pham and Manning 2015; Vulic and Moens 2015), which could be used to model cross-lingual semantic similarity.…”

Section: Comparability Measuresmentioning

confidence: 99%

See 1 more Smart Citation

Measuring bilingual corpus comparability

Gaussier

Yang

2018

Nat. Lang. Eng.

View full text Add to dashboard Cite

Comparable corpora serve as an important substitute for parallel resources in cases of under-resourced language pairs. Previous work mostly aims to find a better strategy to exploit existing comparable corpora, while ignoring the variety in corpus quality. The quality of comparable corpora affects a lot its usability in practice, a fact that has been justified by several studies. However, researchers have not been able to establish a widely accepted and fully validated framework to measure corpus quality. We will thus investigate in this paper a comprehensive methodology to deal with the quality of comparable corpora. To be exact, we will propose several comparability measures and a quantitative strategy to test those measures. Our experiments show that the proposed comparability measure can capture gold-standard comparability levels very well and is robust to the bilingual dictionary used. Moreover, we will show in the task of bilingual lexicon extraction that the proposed measure correlates well with the performance of the real world application.

show abstract

Section: Introductionsupporting

confidence: 68%