Bilingual text, matching using bilingual dictionary and statistics

Utsuro, Takehito; Ikeda, Hiroshi; Masaya, Yamane; Matsumoto, Yūji; Nagao, Makoto

doi:10.3115/991250.991325

Cited by 25 publications

(23 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We adopt a standard strategy to align articles and sentences. First, we use a method based on CLIR to align Japanese and English articles (Collier et al, 1998;Matsumoto and Tanaka, 2002) and then a method based on DP matching to align Japanese and English sentences (Gale and Church, 1993;Utsuro et al, 1994) in these articles. As each of these methods uses existing NLP techniques, we describe them briefly focusing on basic similarity measures, which we will compare with our proposed measures in Section 5.…”

Section: Basic Alignment Methodsmentioning

confidence: 99%

Reliable measures for aligning Japanese-English news articles and sentences

Utiyama¹,

Isahara²

2003

Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03

105

View full text Add to dashboard Cite

We have aligned Japanese and English news articles and sentences to make a large parallel corpus. We first used a method based on cross-language information retrieval (CLIR) to align the Japanese and English articles and then used a method based on dynamic programming (DP) matching to align the Japanese and English sentences in these articles. However, the results included many incorrect alignments. To remove these, we propose two measures (scores) that evaluate the validity of alignments. The measure for article alignment uses similarities in sentences aligned by DP matching and that for sentence alignment uses similarities in articles aligned by CLIR. They enhance each other to improve the accuracy of alignment. Using these measures, we have successfully constructed a largescale article and sentence alignment corpus available to the public.

show abstract

Section: Basic Alignment Methodsmentioning

confidence: 99%

Reliable measures for aligning Japanese-English news articles and sentences

Utiyama¹,

Isahara²

2003

Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03

105

View full text Add to dashboard Cite

show abstract

“…This evaluation was done in an exact condition with the Evaluation 1 to compare how better our proposed method is in comparison with previous studies [13], [9] which used Length, Cognate, and Dictionary for measuring content-based similarity. For this evaluation, we use "top five translations of each of its words" as described in [9].…”

Section: Discussionmentioning

confidence: 99%

“…Some other studies such those of Utsuro [13] and Zhao [17] used a statistical translation lexicon model. However, these works suffer much from the ambiguity of word translations, that cause the error rate problem (i.e.…”

Section: Introductionmentioning

confidence: 99%

Exploiting Non-Parallel Corpora for Statistical Machine Translation

Cuong¹,

Le²,

Nguyen³

et al. 2012

2012 IEEE RIVF International Conference on Computing &Amp; Communication Technologies, Research, Innovation, and Vision for The

View full text Add to dashboard Cite

Abstract-Constructing a corpus of parallel sentence pairs is an important work in building a Statistical Machine Translation system. It impacts deeply how the quality of a Statistical Machine Translation could achieve. The more parallel sentence pairs we use to train the system, the better translation's quality it is. Nowadays, comparable non-parallel corpora become important resources to alleviate scarcity of parallel corpora. The problem here is how to extract parallel sentence pairs automatically but accurately from comparable non-parallel corpora which are usually very "noisy". This paper presents how we can apply the reinforcement learning scheme with our new proposed algorithm for detecting parallel sentence pairs. We specify that from an initial set of parallel sentences in a domain, the proposed model can extract a large number of new parallel sentence pairs from non-parallel corpora resources in different domains, concurrently increasing the system's translation ability by time.

show abstract

“…Comparison, selection and use of sentence alignment algorithms for new language pairs are discussed in Singh [12] . Bilingual text matching using bilingual dictionary and statistics are discussed in [13] .…”

Section: Related Workmentioning

confidence: 99%

Semi-Automatic Parallel Corpora Extraction from Comparable News Corpora

Singh

Bandyopadhyay

2010

Polibits

View full text Add to dashboard Cite

The parallel corpus is a necessary resource in many multi/cross lingual natural language processing applications that include Machine Translation and Cross Lingual Information Retreival. Preparation of large scale parallel corpus takes time and also demands the linguistics skill. In the present work, a technique has been developed that extracts parallel corpus between Manipuri, a morphologically rich and resource constrained Indian language and English from a comparable news corpora collected from the web. A medium sized Manipuri-English bilingual lexicon and another list of Manipuri-English transliterated entities have been developed and used in the present work. Using morphological information for the agglutinative and inflective Manipuri language, the alignment quality based on similarity measure is further improved. A high level of performance is desirable since errors in sentence alignment cause further errors in systems that use the aligned text. The system has been evaluated and error analysis has also been carried out. The technique shows its effectiveness in Manipuri-English language pair and is extendable to other resource constrained, agglutinative and inflective Indian languages.

show abstract

Bilingual text, matching using bilingual dictionary and statistics

Cited by 25 publications

References 13 publications

Reliable measures for aligning Japanese-English news articles and sentences

Reliable measures for aligning Japanese-English news articles and sentences

Exploiting Non-Parallel Corpora for Statistical Machine Translation

Semi-Automatic Parallel Corpora Extraction from Comparable News Corpora

Contact Info

Product

Resources

About