Transferring structural markup across translations using multilingual alignment and projection

Abstract

We present here a method for automatically projecting structural information across translations, including canonical citation structure (such as chapters and sections), speaker information, quotations, markup for people and places, and any other element in TEI-compliant XML that delimits spans of text that are linguistically symmetrical in two languages. We evaluate this technique on two datasets, one containing perfectly transcribed texts and one containing errorful OCR, and achieve an accuracy rate of 88.2% projecting 13,023 XML tags from source documents to their transcribed translations, with an 83.6% accuracy rate when projecting to texts containing uncorrected OCR. This approach has the potential to allow a highly granular multilingual digital library to be bootstrapped by applying the knowledge contained in a small, heavily curated collection to a much larger but unstructured one.

Cross-lingual Annotation Projection in Legal Texts

“…Projection has also been used by Bentivogli and Pianta (2005) to create a parallel version of an existing corpus. The projection of structural information between parallel documents is tackled by Bamman et al (2010), where alignment is performed firstly sentence-wise (1-1) and then word-wise.…”

“…The works of Das andPetrov (2011), Moore (2002), Quan et al (2018), andZamani et al (2016) are all based on symmetric corpora, while imposing a 1-1 alignment is not possible in our domain. Bamman et al (2010), Bentivogli and Pianta (2005), Eger et al (2018), Fossum and Abney (2005), and Yarowsky et al (2001) all address word-level alignment instead of sentence level-alignment. These methods cannot be extended to address sentence-level alignment without introducing an element of subjectivity, for instance, in resolving conflicts of words matched across different sentences.…”

Participatory Philology: Computational Linguistics and the Future of Historical Language Education

“…Work towards generating computational hypotheses for the types of annotation listed above relies on algorithms developed by Saeed Majidi at Tufts (Majidi and Crane, 2013); while development by David Bamman at Carnegie Mellon University informs work towards the automatic markup on documents through aligned translations (Bamman et. al, 2010).…”

Measuring historical word sense variation

“…The size of this collection has made it a natural target for topic modeling [27] (including evaluating topic coherence [30]), where the immensity of the data encourages automatic methods for characterizing it, as well as research into automatic methods for adding structure [3]. More recently, however, researchers have begun to exploit these massive datasets for measuring historical change.…”