Cross-lingual Structure Transfer for Relation and Event Extraction

Abstract

The identification of complex semantic structures such as events and entity relations, already a challenging Information Extraction task, is doubly difficult from sources written in under-resourced and under-annotated languages. We investigate the suitability of crosslingual structure transfer techniques for these tasks. We exploit relation-and event-relevant language-universal features, leveraging both symbolic (including part-of-speech and dependency path) and distributional (including type representation and contextualized representation) information. By representing all entity mentions, event triggers, and contexts into this complex and structured multilingual common space, using graph convolutional networks, we can train a relation or event extractor from source language annotations and apply it to the target language. Extensive experiments on cross-lingual relation and event transfer among English, Chinese, and Arabic demonstrate that our approach achieves performance comparable to state-of-the-art supervised models trained on up to 3,000 manually annotated mentions: up to 62.6% F-score for Relation Extraction, and 63.1% F-score for Event Argument Role Labeling. The event argument role labeling model transferred from English to Chinese achieves similar performance as the model trained from Chinese. We thus find that language-universal symbolic and distributional representations are complementary for cross-lingual structure transfer.

Severing the Edge Between Before and After: Neural Architectures for Temporal Ordering of Events

“…For the future, instead of using the RoBERTa baseline model for the self-training experiments, we could run several iterations by retraining on the data produced by our best self-trained model(s); this could be a good avenue for further improvements. In addition we plan to extend our work by moving to other languages beyond English (we currently have not tried this due to lack of data) using cross-lingual models, (Subburathinam et al, 2019), applying other architectures like CNNs (Nguyen and Grishman, 2015), incorporating tree structure in our models (Miwa and Bansal, 2016) and/or by handling jointly performing event recognition and temporal ordering (Li and Ji, 2014;Katiyar and Cardie, 2017).…”

Event-Centric Natural Language Processing

“…Specifically, for data-driven extraction methods, we will present how constrained learning and structured prediction are incorporated to improve the tasks by enforcing logic consistency among different categories of event-event relations . We will also cover various cross-domain (Huang et al, 2018), cross-lingual (Subburathinam et al, 2019) and cross-media (Li et al, 2020a) structure transfer approaches for event extraction. This part is estimated to be 40 minutes.…”

Syntax-augmented Multilingual BERT for Cross-lingual Transfer

“…Encoding Syntax for Language Transfer Universal language syntax, e.g., part-of-speech (POS) tags, dependency parse structure, and relations are shown to be helpful for cross-lingual transfer (Kozhevnikov and Titov, 2013;Pražák and Konopík, 2017;Wu et al, 2017;Subburathinam et al, 2019;Liu et al, 2019;Xie et al, 2020;Ahmad et al, 2021). Many of these prior works utilized graph neural networks (GNN) to encode the dependency graph structure of the input sequences.…”