TABi: Type-Aware Bi-Encoders for Open-Domain Entity Retrieval

Abstract

Entity retrieval-retrieving information about entity mentions in a query-is a key step in open-domain tasks, such as question answering or fact checking. However, state-of-the-art entity retrievers struggle to retrieve rare entities for ambiguous mentions due to biases towards popular entities. Incorporating knowledge graph types during training could help overcome popularity biases, but there are several challenges: (1) existing type-based retrieval methods require mention boundaries as input, but open-domain tasks run on unstructured text, (2) type-based methods should not compromise overall performance, and (3) type-based methods should be robust to noisy and missing types. In this work, we introduce TABi, a method to jointly train bi-encoders on knowledge graph types and unstructured text for entity retrieval for open-domain tasks. TABi leverages a typeenforced contrastive loss to encourage entities and queries of similar types to be close in the embedding space. TABi improves retrieval of rare entities on the Ambiguous Entity Retrieval (AmbER) sets, while maintaining strong overall retrieval performance on open-domain tasks in the KILT benchmark compared to state-of-the-art retrievers. TABi is also robust to incomplete type systems, improving rare entity retrieval over baselines with only 5% type coverage of the training dataset. We make our code publicly available. 1

Alignment Precedes Fusion: Open-Vocabulary Named Entity Recognition as Context-Type Semantic Matching

“…Architecture We use Dual-Encoder architecture to model context-type pairs similar to entity linking Zhang et al, 2022a,b;Leszczynski et al, 2022). As shown in Figure 2 (a) and (b), Dual-Encoder consists of two independent transformer encoders, called Context-Encoder E ctxt and Type-Encoder E type .…”

Hansel: A Chinese Few-Shot and Zero-Shot Entity Linking Benchmark

“…Wikidata Type system. Prior work demonstrated that types can benefit EL systems (Ling et al, 2015;Raiman and Raiman, 2018;Leszczynski et al, 2022). We introduce a new formulation for coarse and fine entity typing, utilizing rich structural knowledge in Wikidata.…”