Entity linking at the tail

Abstract

Web search is seeing a paradigm shift from keyword based search to an entity-centric organization of web data. To support web search with this deeper level of understanding, a web-scale entity linking system must have 3 key properties: First, its feature extraction must be robust to the diversity of web documents and their varied writing styles and content structures. Second, it must maintain high-precision linking for "tail" (unpopular) entities that is robust to the existence of confounding entities outside of the knowledge base and entity profiles with minimal information. Finally, the system must represent large-scale knowledge bases with a scalable and powerful feature representation. We have built and deployed a web-scale unsupervised entity linking system for a commercial search engine that addresses these requirements by combining new developments in sparse signal recovery to identify the most discriminative features from noisy, free-text web documents; explicit modeling of out-of-knowledge-base entities to improve precision at the tail; and the development of a new phrase-unigram language model to efficiently capture high-order dependencies in lexical features. Using a knowledge base of 100M unique people from a popular social networking site, we present experimental results in the challenging domain of people-linking at the tail, where most entities have limited web presence. Our experimental results show that this system substantially improves on the precision-recall tradeoff over baseline methods, achieving precision over 95% with recall over 60%.

Cross-Lingual Transfer in Zero-Shot Cross-Language Entity Linking

“…This could be accomplished by adding a document-level representation or by leveraging other mentions in the document. English-focused work on rare entity performance (Orr et al, 2020;Jin et al, 2014) may provide additional direction.…”

Entity Resolution in the Web of Data

“…To serve this navigational request ⁵¹A process known as named-entity extraction , Etzioni et al, 2005, Hoffart et al, 2013 and disambiguation [Jin et al, 2014]. To serve this navigational request ⁵¹A process known as named-entity extraction , Etzioni et al, 2005, Hoffart et al, 2013 and disambiguation [Jin et al, 2014].…”

LinkNER: Linking Local Named Entity Recognition Models to Large Language Models using Uncertainty

“…Recent studies show that NER models face challenges in predicting entities that are not encountered during training [11,25,34], encompassing both Out-of-Vocabulary (OOV) and Out-of-Domain (OOD) entities. These challenges frequently arise in web social media and the medical domain [7,18]. Consequently, NER models' ability to detect and classify decreases when they encounter these entities, which we attribute to the "lack of knowledge" of fine-tuned small models.…”