Improving Machine Translation Performance by Exploiting Non-Parallel Corpora

Munteanu, Dragos Stefan; Marcu, Daniel

doi:10.1162/089120105775299168

Cited by 217 publications

(108 citation statements)

References 19 publications

Supporting

Mentioning

107

Contrasting

Order By: Relevance

“…Translations of in-domain terms can be mined from comparable corpora, i.e. texts that are not strictly parallel but deal with the same topic [29,30]. Bertoldi and Federico [31] exploit large amounts of in-domain monolingual data to create synthetic parallel training corpora.…”

Section: Domain Adaptationmentioning

confidence: 99%

Adaptation of machine translation for multilingual information retrieval in the medical domain

Pecina¹,

Dušek²,

Goeuriot³

et al. 2014

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

Objective. We investigate machine translation (MT) of user search queries in the context of cross-lingual information retrieval (IR) in the medical domain. The main focus is on techniques to adapt MT to increase translation quality; however, we also explore MT adaptation to improve effectiveness of cross-lingual IR.Methods and Data. Our MT system is Moses, a state-of-the-art phrase-based statistical machine translation system. The IR system is based on the BM25 retrieval model implemented in the Lucene search engine. The MT techniques employed in this work include in-domain training and tuning, intelligent training data selection, optimization of phrase table configuration, compound splitting, and exploiting synonyms as translation variants. The IR methods include morphological normalization and using multiple translation variants for query expansion. The experiments are performed and thoroughly evaluated on three language pairs: Czech-English, German-English, and French-English. MT quality is evaluated on data sets created within the Khresmoi project and IR effectiveness is tested on the CLEF eHealth 2013 data sets.Results. The search query translation results achieved in our experiments are outstanding -our systems outperform not only our strong baselines, but also Google Translate and Microsoft Bing Translator in direct comparison carried out on all the language pairs. The baseline BLEU scores increased from 26.59 to 41.45 for Czech-English, from 23.03 to 40.82 for German-English, and from 32.67 to 40.82 for French-English. This is a 55% improvement on average. In terms of the IR performance on this particular test collection, a significant improvement over the baseline is achieved only for French-English. For Czech-English and German-English, the increased MT quality does not lead to better IR results.Conclusions. Most of the MT techniques employed in our experiments improve MT of medical search queries. Especially the intelligent training data selection proves to be very successful for domain adaptation of MT. Certain improvements are also obtained from German compound splitting on the source language side. Translation quality, however, does not appear to correlate with the IR performance -better translation does not necessarily yield better retrieval. We discuss in detail the contribution of the individual techniques and state-of-the-art features and provide future research directions.

show abstract

Section: Domain Adaptationmentioning

confidence: 99%

Adaptation of machine translation for multilingual information retrieval in the medical domain

Pecina¹,

Dušek²,

Goeuriot³

et al. 2014

Artificial Intelligence in Medicine

View full text Add to dashboard Cite

show abstract

“…In their later seminal paper, Munteanu and Marcu (2005) improved their method by training a maximum entropy classifier which for a given pair of sentences can reliably determine whether or not they are translations of each other. They also showed empirically that a statistical MT system can be built from scratch by starting with a small parallel corpus of only 100,000 words and by expanding it using parallel segments as extracted from pairs of the very large Gigaword-Corpora (Arabic-English and Chinese-English).…”

Section: Mining Parallel Segments From Comparable Corporamentioning

confidence: 99%

Recent advances in machine translation using comparable corpora

Rapp¹,

Sharoff²,

Zweigenbaum³

2016

Nat. Lang. Eng.

View full text Add to dashboard Cite

This paper highlights some of the recent developments in the field of machine translation using comparable corpora. We start by updating previous definitions of comparable corpora and then look at bilingual versions of continuous vector space models. Recently, neural networks have been used to obtain latent context representations with only few dimensions which are often called word embeddings. These promising new techniques cannot only be applied to parallel but also to comparable corpora. Subsequent sections of the paper discuss work specifically targeting at machine translation using comparable corpora, as well as work dealing with the extraction of parallel segments from comparable corpora. Finally, we give an overview on the design and the results of a recent shared task on measuring document comparability across languages.

show abstract

“…Statistical machine translation, uses manually translated data in the forms of parallel sentences to learn translation patterns by statistical means. There has been extensive work focusing in finding parallel documents [14] and aligning sentences in fairly parallel corpora [8] and even non-parallel corpora [9]. [10] presents an approach to find sub-sentential segments from comparable corpora.…”

Section: Related Workmentioning

confidence: 99%

Cross-Lingual Text Fragment Alignment Using Divergence from Randomness

Yahyaei

Bonzanini

Roelleke

2011

String Processing and Information Retrieval

View full text Add to dashboard Cite

Abstract. This paper describes an approach to automatically align fragments of texts of two documents in different languages. A text fragment is a list of continuous sentences and an aligned pair of fragments consists of two fragments in two documents, which are content-wise related. Cross-lingual similarity between fragments of texts is estimated based on models of divergence from randomness. A set of aligned fragments based on the similarity scores are selected to provide an alignment between sections of the two documents. Similarity measures based on divergence show strong performance in the context of cross-lingual fragment alignment in the performed experiments.

show abstract

Improving Machine Translation Performance by Exploiting Non-Parallel Corpora

Cited by 217 publications

References 19 publications

Adaptation of machine translation for multilingual information retrieval in the medical domain

Adaptation of machine translation for multilingual information retrieval in the medical domain

Recent advances in machine translation using comparable corpora

Cross-Lingual Text Fragment Alignment Using Divergence from Randomness

Contact Info

Product

Resources

About