Learning Dynamic Context Augmentation for Global Entity Linking

Abstract

Despite of the recent success of collective entity linking (EL) methods, these "global" inference methods may yield sub-optimal results when the "all-mention coherence" assumption breaks, and often suffer from high computational cost at the inference stage, due to the complex search space. In this paper, we propose a simple yet effective solution, called Dynamic Context Augmentation (DCA), for collective EL, which requires only one pass through the mentions in a document. DCA sequentially accumulates context information to make efficient, collective inference, and can cope with different local EL models as a plugand-enhance module. We explore both supervised and reinforcement learning strategies for learning the DCA model. Extensive experiments 1 show the effectiveness of our model with different learning settings, base models, decision orders and attention mechanisms.

Improving Entity Linking by Modeling Latent Entity Type Information

“…By integrating the BERTbased entity similarity, our proposed model can correct 124 out of 185 (67.03%) type error cases of the baseline model which demonstrates that we correct more than two third of the type errors produced by the baseline. We have further examined and categorized the remaining 61 type error cases into three categories: (i) Due to prior: golden entities with very lowp(e|m i ) prior, (ii) Due to global: both the local context score and prior score support predicting the golden Methods AIDA-B MSNBC AQUAINT ACE2004 CWEB WIKI Avg Ganea and Hofmann (2017) 92.22 ± 0.14 93.7 ± 0.1 88.5 ± 0.4 88.5 ± 0.3 77.9 ± 0.1 77.5 ± 0.1 85.22 Le and Titov (2018) 93.07 ± 0.27 93.9 ± 0.2 88.3 ± 0.6 89.9 ± 0.8 77.5 ± 0.1 78.0 ± 0.1 85.51 Yang et al (2019) 94.64 ± 0.20 94.6 ± 0.2 87.4 ± 0.5 89.4 ± 0.4 73.5 ± 0.1 78.2 ± 0.1 84.62 BERT-Entity-Sim (local & global) 93.54 ± 0.12 93.4 ± 0.1 89.8 ± 0.4 88.9 ± 0.7 77.9 ± 0. entity, but the overall score supports predicting incorrect entity due to global modeling, (iii) Due to local context: the local context score misleads the model predicting the wrong entity, this is potentially due to the mention context can be misleading, e.g. a document discussing cricket will favor resolving the mention "Australian" in context "impressed by the positive influence of Australian coach Dave Gilbert" to the entity AUSTRALIA NATIONAL CRICKET TEAM instead of the gold entity AUSTRALIA.…”

“…Better Global Model In order to investigate whether better global model can further boost the performance of our model, we incorporate the recent proposed Dynamic Context Augmentation (DCA) 15 (Yang et al 2019). DCA is a global entity linking model featuring better efficiency and effectiveness than that of Ganea and Hofmann (2017) by breaking the "all-mention coherence" assumption.…”

“…This shows that better global model indeed can further boost the performance of our proposed model. Notice that Yang et al (2019) achieves a high performance on in-domain AIDA-CoNLL test set, but doesn't perform well on five out-domain test sets (even inferior than Ganea and Hofmann (2017)). We found that Yang et al (2019) includes an explicit type similarity which is based on a typing system 16 trained with AIDA-train NER annotation.…”

“…Notice that Yang et al (2019) achieves a high performance on in-domain AIDA-CoNLL test set, but doesn't perform well on five out-domain test sets (even inferior than Ganea and Hofmann (2017)). We found that Yang et al (2019) includes an explicit type similarity which is based on a typing system 16 trained with AIDA-train NER annotation. This explicit type similarity feature is tailored for AIDA-CoNLL data set and doesn't achieve good generalization performance on out-domain test sets.…”

Evaluating the Impact of Knowledge Graph Context on Entity Disambiguation Models

“…Our best performance can then be attributed to the quality of textual context learned by the transformers as well as the optimal choice of KG-triples context. Generalizing KG Context: We induced 1-hop KG context in DCA-SL model [12] for candidate entities. The replacement of the unstructured Wikipedia description with structured KG triple context containing entity aliases, entity types, consolidated entity description, etc.…”