Zero-shot Entity Linking with Efficient Long Range Sequence Modeling

Yao, Zonghai; Cao, Liangliang; Pan, Huapu

doi:10.18653/v1/2020.findings-emnlp.228

“…BLINK (Wu et al, 2020) proposes a bi-encoder to encode the descriptions and enhance the bi-encoder by distilling the knowledge from the cross-encoder. Yao et al (2020) repeats the position embedding to solve the long-range modeling problem in entity descriptions. Zhang and Stratos (2021) demonstrates that hard negatives can enhance the contrast when training an EL model.…”

Section: Related Workmentioning

confidence: 99%

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Ma¹,

Jiang²,

Bach³

et al. 2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural language processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and efficient framework to nominate candidates if entities are only identified by descriptions. However, they ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method. Our method achieves the state-ofthe-art performance on ZESHEL and improves the quality of candidates on three standard Entity Linking datasets 1 .

show abstract

“…As an encoder BERT model is selected. While state-of-the-art models in Zero-shot EL (Logeswaran et al, 2019;Yao et al, 2020) focus on the CR phase, Wu et al (2020) are the only to propose a different to traditional IR approach for CG. Our focus is to further push the boundaries of the CG phase and set a higher performance threshold to CR and EL overall.…”

Section: State-of-the-art Cg Modelsmentioning

confidence: 99%

“…Most EL systems consist of two subsystems: Candidate Generation (CG), where for each entity mention the system detects entities related to the mention and document, and Candidate Ranking (CR) where the system chooses the most probable entity link among the found candidates. Most state-of-the-art models (Logeswaran et al, 2019;Li et al, 2020;Yao et al, 2020) rely on traditional frequency-based CG and focus on building robust candidate rankers using cross-encoders to jointly encode mention and entity candidate descriptions. However, the memory-intensive CR phase depends on the set of candidates provided by CG.…”

Section: Introductionmentioning

confidence: 99%

Improving Zero-Shot Entity Retrieval through Effective Dense Representations

Partalidou¹,

Christou²,

Tsoumakas³

2021

Preprint

0

View full text Add to dashboard Cite

Entity Linking (EL) seeks to align entity mentions in text to entries in a knowledge-base and is usually comprised of two phases: candidate generation and candidate ranking. While most methods focus on the latter, it is the candidate generation phase that sets an upper bound to both time and accuracy performance of the overall EL system. This work's contribution is a significant improvement in candidate generation which thus raises the performance threshold for EL, by generating candidates that include the gold entity in the least candidate set (top-K). We propose a simple approach that efficiently embeds mention-entity pairs in dense space through a BERT-based biencoder. Specifically, we extend (Wu et al., 2020) by introducing a new pooling function and incorporating entity type side-information. We achieve a new state-of-the-art 84.28% accuracy on top-50 candidates on the Zeshel dataset, compared to the previous 82.06% on the top-64 of (Wu et al., 2020). We report the results from extensive experimentation using our proposed model on both seen and unseen entity datasets. Our results suggest that our method could be a useful complement to existing EL approaches.

show abstract

“…BLINK (Wu et al, 2020) proposes a bi-encoder to encode the descriptions and enhance the bi-encoder by distilling the knowledge from the cross-encoder. Yao et al (2020) repeats the position embedding to solve the long-range modeling problem in entity descriptions. Zhang and Stratos (2021) demonstrates that hard negatives can enhance the contrast when training an EL model.…”

Section: Related Workmentioning

confidence: 99%

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Ma¹,

Jiang²,

Bach³

et al. 2021

Preprint

0

View full text Add to dashboard Cite

Entity retrieval, which aims at disambiguating mentions to canonical entities from massive KBs, is essential for many tasks in natural language processing. Recent progress in entity retrieval shows that the dual-encoder structure is a powerful and efficient framework to nominate candidates if entities are only identified by descriptions. However, they ignore the property that meanings of entity mentions diverge in different contexts and are related to various portions of descriptions, which are treated equally in previous works. In this work, we propose Multi-View Entity Representations (MuVER), a novel approach for entity retrieval that constructs multi-view representations for entity descriptions and approximates the optimal view for mentions via a heuristic searching method. Our method achieves the state-ofthe-art performance on ZESHEL and improves the quality of candidates on three standard Entity Linking datasets 1 .

show abstract

Zero-shot Entity Linking with Efficient Long Range Sequence Modeling

Cited by 16 publications

References 15 publications

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Improving Zero-Shot Entity Retrieval through Effective Dense Representations

MuVER: Improving First-Stage Entity Retrieval with Multi-View Entity Representations

Contact Info

Product

Resources

About