A New Entity Salience Task with Millions of Training Examples

Abstract

Although many NLP systems are moving toward entity-based processing, most still identify important phrases using classical keyword-based approaches. To bridge this gap, we introduce the task of entity salience: assigning a relevance score to each entity in a document. We demonstrate how a labeled corpus for the task can be automatically generated from a corpus of documents and accompanying abstracts. We then show how a classifier with features derived from a standard NLP pipeline outperforms a strong baseline by 34%. Finally, we outline initial experiments on further improving accuracy by leveraging background knowledge about the relationships between entities.

Ranking Entities for Web Queries Through Text and Knowledge

“…The question of ranking entities from documents, which we implicitly touch by ranking entities from the search result documents, is addressed in recent work from Dunietz and Gillick, who define the task of "entity salience [as] assigning a relevance score to each entity in a document" [13]. However, even though our method could also be used for entity ranking at the document level, we take here a query-centric view of the problem.…”

“…This makes it possible for us to leverage a Wikipedia-based resource such as DBpedia in a straightforward way, and evaluate the contribution of a wide-coverage knowledge base for our problem. An 'open' definition of entities, like the one found in [13], instead, opens up new problems such as how to detect and include new entities into the background knowledge base: although recently there have been attempts to address this problem at web scale [20], we leave this issue for future work.…”

“…Most tracks build upon a definition of entities based on Wikipedia, i.e., every Wikipedia article is a valid entity, which we also use. However, alternative or broader definitions are possible (see, e.g., [2,13]). The focus on type queries led to the development of systems that leveraged Wikipedia category system -e.g., category overlap [32], or measures capturing the strength of association between terms and category labels [27].…”

“…Starting with the Initiative for the Evaluation of XML Retrieval (INEX) [11] and TREC initiatives [2], the task of entity ranking has gained momentum, and a variety of different approaches have been developed, ranging from natural language interface for XML retrieval and Question Answering over Linked Data ( [15,39,43], inter alia) all the way through open-domain, document-based entity search [13]. Evaluation exercises on entity ranking systems within the INEX evaluation forum have taken place since 2006.…”

Identification and Characterization of Newsworthy Verbs in World News

“…Summaries are intended to convey important information while omitting the less important pieces, so words that are important in a newsworthy sense will occur more frequently in summaries. The same data and intuition was used recently to develop a large corpus for determining entity salience (Dunietz and Gillick, 2014).…”