Concept-Based Information Retrieval Using Explicit Semantic Analysis

Abstract

Information retrieval systems traditionally rely on textual keywords to index and retrieve documents. Keyword-based retrieval may return inaccurate and incomplete results when different keywords are used to describe the same concept in the documents and in the queries. Furthermore, the relationship between these related keywords may be semantic rather than syntactic, and capturing it thus requires access to comprehensive human world knowledge. Concept-based retrieval methods have attempted to tackle these difficulties by using manually built thesauri, by relying on term cooccurrence data, or by extracting latent word relationships and concepts from a corpus. In this article we introduce a new concept-based retrieval approach based on Explicit Semantic Analysis (ESA), a recently proposed method that augments keywordbased text representation with concept-based features, automatically extracted from massive human knowledge repositories such as Wikipedia. Our approach generates new text features automatically, and we have found that high-quality feature selection becomes crucial in this setting to make the retrieval more focused. However, due to the lack of labeled data, traditional feature selection methods cannot be used, hence we propose new methods that use self-generated labeled training data. The resulting system is evaluated on several TREC datasets, showing superior performance over previous state-of-the-art results.

What’s New? Analysing Language-Specific Wikipedia Entity Contexts to Support Entity-Centric News Retrieval

“…Examples include information retrieval [4,14,18], named entity disambiguation [1,2,7,8,11,12], text classification [25] and entity ranking [10]. To extract the content of an entity context, many researches directly used the Wikipedia article describing the entity [1,2,8,9,14,[25][26][27]; some works extended the article with all the other Wikipedia articles linked to the Wikipedia article describing the entity [6,7,12]; while some only considered the first paragraph of the Wikipedia article describing the entity [2].…”

“…As for incorporating the Wikipedia knowledge in information retrieval applications, [4,15,18] applied concept-based approaches that mapped both the documents and queries to the Wikipedia concept space; [14,23] focused only on query extension; [20,24] focused only on mapping documents to Wikipedia concept space. To retrieve documents that did not explicitly mention the query entity by name, but were still relevant to the query entity, we chose to map both the query and the documents to the aspect space.…”

Handbook of Technical Communication

“…This includes the notion of the web of data (cf. the linked data paradigm (Chiarcos, Nordhoff, and Hellmann, 2012)) as well as models of social and emergent semantics (Steels and Hanappe, 2006;Gabrilovich and Markovitch, 2009;Egozi, Markovitch, and Gabrilovich, 2011). In this sense, Chapter 16 can be read as an overview of the development of social complexity in web-based communication starting from mass communication (where users tend to be passive recipients of static web content), and advancing now to the area of true n : m-communication as exemplified by collaborative writing and microblogging (where users perform both roles as authors and recipients).…”

Ranking Entities for Web Queries Through Text and Knowledge

“…increasing coverage and complexity [42,18]. These can be leveraged either as sources of information to be presented directly to the user or as background knowledge sources to be used by retrieval systems to improve the retrieval process itself [8,14]. But despite the sophistication of existing search engines, the former task of satisfying the user information need by presenting an entity and its related description is still tackled in a very basic way.…”