Weakly Supervised Cross-Lingual Named Entity Recognition via Effective Annotation and Representation Projection

Abstract

The state-of-the-art named entity recognition (NER) systems are supervised machine learning models that require large amounts of manually annotated data to achieve high accuracy. However, annotating NER data by human is expensive and time-consuming, and can be quite difficult for a new language. In this paper, we present two weakly supervised approaches for cross-lingual NER with no human annotation in a target language. The first approach is to create automatically labeled NER data for a target language via annotation projection on comparable corpora, where we develop a heuristic scheme that effectively selects goodquality projection-labeled data from noisy data. The second approach is to project distributed representations of words (word embeddings) from a target language to a source language, so that the sourcelanguage NER system can be applied to the target language without re-training. We also design two co-decoding schemes that effectively combine the outputs of the two projection-based approaches. We evaluate the performance of the proposed approaches on both in-house and open NER data for several target languages. The results show that the combined systems outperform three other weakly supervised approaches on the CoNLL data.

Adversarial Learning with Contextual Embeddings for Zero-resource Cross-lingual Classification and NER

“…Regard- less, the BERT zero-resource performance far exceeds the results published in previous work. Mayhew et al (2017) and Ni et al (2017) do use some cross-lingual resources (like bilingual dictionaries) in their experiments, but it appears that BERT with multilingual pretraining performs better, even though it does not have access to crosslingual information. Table 3: Median cosine similarity between the mean-pooled BERT embeddings of MLDoc English documents and their translations, with and without language-adversarial training.…”

Entity Projection via Machine Translation for Cross-Lingual NER

“…This requires first projecting annotations from the source data to the (unlabeled) target data. Many approaches in this category rely upon parallel corpora (Yarowsky et al, 2001;Zeman and Resnik, 2008;Ehrmann et al, 2011;Fu et al, 2011;Ni et al, 2017), first annotating the source data using a trained model and then projecting the annotations. Only a few works explore the use of MT to first translate a gold annotated corpus to obtain a synthetic parallel corpus and then project annotations (Tiedemann et al, 2014).…”

“…When projecting annotations, one encounters the problem of word alignment. Most of the existing works (Yarowsky et al, 2001;Shah et al, 2010;Ni et al, 2017) rely upon unsupervised alignment models from statistical MT literature, such as IBM Models 1-6 (Brown et al, 1993;Och and Ney, 2003). Other works focus on low-resource settings (Mayhew et al, 2017;Xie et al, 2018) perform translation word-by-word or phrase-by-phrase, and thus do not need to perform word alignment.…”

“…Since this algorithm can produce multiple matches for a given source entity, we postprocess the alignments produced by this algorithm and select the longest match and then project tags in the same way as our method. Our third baseline is Ni et al (2017) (Co-decoding), who use a co-decoding scheme on two different NER models. We also compare our method with Polyglot-NER (Al-Rfou et al, 2015) who use Wikipedia links to project entities.…”

Low-Resource Sequence Labeling via Unsupervised Multilingual Contextualized Representations

“…In annotation projection approaches, parallel or comparable corpora are commonly used (Yarowsky et al, 2001;Ehrmann et al, 2011;Das and Petrov, 2011;Li et al, 2012;Täckström et al, 2013;Wang and Manning, 2014;Ni et al, 2017). The source language sentences of parallel corpora are first annotated either manually or by a pretrained tagger.…”