Automatic extraction of translations from web-based bilingual materials

Abstract

Abstract. This paper describes the framework of the StatCan Daily Translation Extraction System (SDTES), a computer system that maps and compares webbased translation texts of Statistics Canada (StatCan) news releases in the StatCan publication The Daily. The goal is to extract translations for translation memory systems, for translation terminology building, for cross-language information retrieval and for corpus-based machine translation systems. Three years of officially published statistical news release texts at www.statcan.ca were collected to compose the StatCan Daily data bank. The English and French texts in this collection were roughly aligned using the Gale-Church statistical algorithm. After this, boundary markers of text segments and paragraphs were adjusted and the Gale-Church algorithm was run a second time for a more fine-grained text segment alignment. To detect misaligned areas of texts and to prevent mis-matched translation pairs from being selected, key textual and structural properties of the mapped texts were automatically identified and used as anchoring features for comparison and misalignment detection. Results show that SDTES is very efficient in extracting translations from Daily texts, and very accurate in identifying mismatched translations. With parameters tuned, the text-mapping part can be used to align officially published bilingual government web-site materials; and the text-comparing component can be applied in pre-publication translation quality control and in evaluating the results of statistical machine translation systems.

Parallel text mapping of web-based bilingual corpus materials

“…The collections will grow and expand in the future. In addition, it also includes a large collection of officially released Canada Hansard materials from the 35 th Parliament in 1994 to the 40 th Parliament at the end of 2008.All the text segments in the Hansard collection are aligned and filtered using the algorithms that are extended from those described in our previous work(Zhu et al 2007). When we search for translations of frequently used words (stop words not included), returned hits of aligned translation pairs can easily exceed the 10,000 mark.…”