Reliable measures for aligning Japanese-English news articles and sentences

Utiyama, Masao; Isahara, Hitoshi

doi:10.3115/1075096.1075106

Cited by 105 publications

(71 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They report improvements in word alignments using parallel sentences discovered by their method. Utiyama and Isahara (2003) use CLIR techniques and dynamic programming (DP) to extract sentences from an English-Japanese comparable news corpus. They identify similar article pairs, and then, treating these pairs as parallel texts, align their sentences on a sentence pair similarity score and use DP to find the least-cost alignment over the document pair.…”

Section: Related Workmentioning

confidence: 99%

Parallel sentence generation from comparable corpora for improved SMT

Rauf¹,

Schwenk²

2011

Machine Translation

View full text Add to dashboard Cite

A parallel corpus is an essential resource for statistical machine translation (SMT) but is often not available in the required amounts for all domains and languages. An approach is presented here which aims at producing parallel corpora from available comparable corpora. An SMT system is used to translate the source-language part of a comparable corpus and the translations are used as queries to conduct information retrieval from the target-language side of the comparable corpus. Simple filters are then used to score the SMT output and the IR-returned sentence with the filter score defining the degree of similarity between the two. Using SMT system output gives the benefit of trying to correct one of the common errors by sentence tail removal. The approach was applied to Arabic-English and French-English systems using comparable news corpora and considerable improvements were achieved in the BLEU score. We show that our approach is independent of the quality of the SMT system used to make the queries, strengthening the claim of applicability of the approach for languages and domains with limited parallel corpora available to start with. We compare our approach with one of the earlier approaches and show that our approach is easier to implement and gives equally good improvements.

show abstract

Section: Related Workmentioning

confidence: 99%

Parallel sentence generation from comparable corpora for improved SMT

Rauf¹,

Schwenk²

2011

Machine Translation

View full text Add to dashboard Cite

show abstract

“…The students in the DDL class worked in pairs to explore targeted recurring grammatical structures in a bilingual newspaper corpus (Utiyama & Isahara, 2003), using a parallel concordancer (Paraconc, 2004). Following guidelines on a worksheet, they searched for and examined basic grammatical structures, formed hypotheses about the structures, discussed these with their partners, and recorded their findings.…”

Section: The 2008 Studymentioning

confidence: 99%

An Effective Way to Use Corpus Exercises to Learn Grammar Basics in English

Oghigian

Chujo

2010

LEiA

View full text Add to dashboard Cite

Computer-based data driven learning (DDL) is becoming gradually more prevalent in L2 classrooms, and studies have shown that DDL exercises can be effective even at the beginner level in teaching vocabulary and grammar (Chujo & Oghigian, 2006. In addition, paperbased DDL studies are gaining ground (Boulton, 2008b), so that even without computers, students are able to explore concordance lines to identify recurring grammatical features and word derivations and to learn how to induce generalizations from the samples to the language in general. In this study, the researchers have developed a series of computer-based and paperbased DDL exercises focused on grammar basics and using a four-step approach that is both inductive and deductive. Course evaluations indicate statistically significant gains in most areas, and students report an overall positive rating on questionnaires.The benefits of data driven learning (DDL) or corpus-based exercises in the L2 classroom are widely recognized. Leech tells us that "… the corpus, as an information source, fits in very well with the dominant trend…from teaching as imparting knowledge to teaching as mediated learning" (1997, p. 2); others point to advantages of using DDL: authenticity; active, studentcentered learning where the learner has control of the learning process; and the powerful and motivating discovery aspect in which the learner acts as researcher (Braun, 2008;Huang, 2008;Hunston, 2002;Johns, 1994;O'Keefe, McCarthy, & Carter, 2007;Stevens, 1995). There is no question that corpus-based learning is gaining ground in second language acquisition; however, Boulton's (2008a) survey of 50 DDL studies shows only four at the beginner level. Corpus use at the beginning level, either directly on computers or with paper-based material, is the focus of this study.In the first section of this paper, several considerations for using DDL in the beginner level L2 classroom are provided. Next, examples of computer-based and paper-based vocabulary and grammar exercises are given in the second section. These exercises have been used in the Nihon University Study, described in the third section. In the final section, the advantages and disadvantages of computer-based and paper-based corpus exercises are discussed.

show abstract

“…A bottleneck in statistical machine translation is the scarceness of parallel resources for many language pairs and domains. Previous research has shown that this bottleneck can be reduced by utilizing parallel portions found within comparable corpora (Utiyama and Isahara, 2003;Munteanu et al, 2004;AbdulRauf and Schwenk, 2009). These are useful for many purposes, including automatic terminology extraction and the training of statistical MT systems.…”

Section: Introductionmentioning

confidence: 99%

Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora

Zweigenbaum¹,

Sharoff²,

Rapp³

2017

Proceedings of the 10th Workshop on Building and Using Comparable Corpora

View full text Add to dashboard Cite

This paper presents the BUCC 2017 shared task on parallel sentence extraction from comparable corpora. It recalls the design of the datasets, presents their final construction and statistics and the methods used to evaluate system results. 13 runs were submitted to the shared task by 4 teams, covering three of the four proposed language pairs: French-English (7 runs), German-English (3 runs), and Chinese-English (3 runs). The best F-scores as measured against the gold standard were 0.84 (GermanEnglish), 0.80 (French-English), and 0.43 (Chinese-English). Because of the design of the dataset, in which not all gold parallel sentence pairs are known, these are only minimum values. We examined manually a small sample of the false negative sentence pairs for the most precise FrenchEnglish runs and estimated the number of parallel sentence pairs not yet in the provided gold standard. Adding them to the gold standard leads to revised estimates for the French-English F-scores of at most +1.5pt. This suggests that the BUCC 2017 datasets provide a reasonable approximate evaluation of the parallel sentence spotting task.

show abstract

Reliable measures for aligning Japanese-English news articles and sentences

Cited by 105 publications

References 7 publications

Parallel sentence generation from comparable corpora for improved SMT

Parallel sentence generation from comparable corpora for improved SMT

An Effective Way to Use Corpus Exercises to Learn Grammar Basics in English

Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora

Contact Info

Product

Resources

About