Attention-Based Joint Entity Linking with Entity Embedding

Abstract

Entity linking (also called entity disambiguation) aims to map the mentions in a given document to their corresponding entities in a target knowledge base. In order to build a high-quality entity linking system, efforts are made in three parts: Encoding of the entity, encoding of the mention context, and modeling the coherence among mentions. For the encoding of entity, we use long short term memory (LSTM) and a convolutional neural network (CNN) to encode the entity context and entity description, respectively. Then, we design a function to combine all the different entity information aspects, in order to generate unified, dense entity embeddings. For the encoding of mention context, unlike standard attention mechanisms which can only capture important individual words, we introduce a novel, attention mechanism-based LSTM model, which can effectively capture the important text spans around a given mention with a conditional random field (CRF) layer. In addition, we take the coherence among mentions into consideration with a Forward-Backward Algorithm, which is less time-consuming than previous methods. Our experimental results show that our model obtains a competitive, or even better, performance than state-of-the-art models across different datasets.

Cluster-based mention typing for named entity disambiguation

“…When it comes to deep learning approaches, Phan et al (2017) employed Long Short-Term Memory (LSTM) networks with attention mechanism. Ganea and Hofmann (2017) used attention mechanism over local context windows to spot important words and Liu et al (2019) expanded this to the important spans with conditional random fields. While these approaches used neural networks with attention mechanism to model the named entities and mentions together and pick the best matching candidate entity, we used a simple LSTM to model the type prediction of the mentions only and then used that information as an extra clue for a simple feed-forward neural network-based ranking model.…”

“…WordNet uses the synset records to define a hypernym hierarchy between them to reflect which synset is a type of which other. YAGO v3.1 (Mahdisoltani et al 2015) uses those synsets as category labels to represent certain types of named entities, such as person, musician, and city. Moreover, YAGO extends the WordNet synsets with its own specialized synsets, prefixed "wikicat" (e.g., wordnet_person_100007846 is a hypernym of wikicat_Men in the YAGO synset taxonomy).…”

A Multi-Angle Bidirectional Interaction Model for Entity Linking

“…These neural-network-based entity linking models [17] [18] [19] can use the surrounding context information to address the linking issue. The global consistency information is also introduced together with the local model for better disambiguation [20][21] [22][23] [24][25] [26][27] [28] These models improve the performance of term representation and linking. However, these methods represent mentions and candidate entities separately, ignoring the interaction information between them.…”