Ranking Entities for Web Queries Through Text and Knowledge

Abstract

When humans explain complex topics, they naturally talk about involved entities, such as people, locations, or events. In this paper, we aim at automating this process by retrieving and ranking entities that are relevant to understand free-text web-style queries like Argentine British relations, which typically demand a set of heterogeneous entities with no specific target type like, for instance, Falklands_-War or Margaret_Thatcher, as answer. Standard approaches to entity retrieval rely purely on features from the knowledge base. We approach the problem from the opposite direction, namely by analyzing web documents that are found to be query-relevant. Our approach hinges on entity linking technology that identifies entity mentions and links them to a knowledge base like Wikipedia. We use a learning-to-rank approach and study different features that use documents, entity mentions, and knowledge base entities -thus bridging document and entity retrieval. Since established benchmarks for this problem do not exist, we use TREC test collections for document ranking and collect custom relevance judgments for entities. Experiments on TREC Robust04 and TREC Web13/14 data show that: i) single entity features, like the frequency of occurrence within the top-ranked documents, or the query retrieval score against a knowledge base, perform generally well; ii) the best overall performance is achieved when combining different features that relate an entity to the query, its document mentions, and its knowledge base representation.

Building Entity-Centric Event Collections

“…Combinations of knowledge base retrieval and entity linking methods have been studied for web search queries both for entity ranking tasks [38,42] as well as document ranking tasks [12,29,48]. Our work builds on these ideas for the purpose of creating event collections.…”

Finding Relevant Relations in Relevant Documents

“…We augment existing test collections for document-relevance and entity-relevance with assessments on correctness and query-relevance of facts; and make the dataset publicly available.We base our analysis on the collection of test queries from the TREC Web track and documents from the ClueWeb12 corpus, which includes relevance assessments for documents, and the REWQ gold standard of query-relevant entities [8]. 6 The TREC Web track studies queries which fall into one of two categories: the query either constitutes an entity which is neither a person, organization, nor location i.e., "Raspberry Pi", or the query is about a concept or entity in a particular context, such as "Nicolas Cage movies".…”

“…In order to study this implication, we make use of the REWQ test collection on entity relevance [8] by studying the subset of the 108 correct facts where relevance assessments exist for both entities. Due to pooling strategies, this subset has a higher precision of 0.722.…”

“…go beyond previous work on identifying relevant entities for Web queries [8] (where relations between entities were not considered), and query-agnostic knowledge base population (where determining fact relevance is not part of the task).…”

Improving Entity Ranking for Keyword Queries

“…Schuhmacher et al introduced a new dataset based on the TREC Robust04 and Web13/14 data [25], and evaluated a learning-to-rank approach based upon features of entities linked in pseudorelevant documents. These entity links could be compared to the sort of document-person co-occurrence or authorship information used in expert search.…”