Automatic query wefinement using lexical affinities with maximal information gain

Abstract

This work describes an automatic query refinement technique, which focuses on improving precision of the top ranked documents. The terms used for refinement are lexical affinities (LAs), pairs of closely related words which contain exactly one of the original query terms. Adding these terms to the query is equivalent to re-ranking search results, thus, precision is improved while recall is preserved. We describe a novel method that selects the most "informative" LAs for refinement, namely, those LAs that best separate relevant documents from irrelevant documents in the set of results. The information gain of candidate LAs is determined using unsupervised estimation that is based on the scoring function of the search engine. This method is thus fully automatic and its quality depends on the quality of the scoring function. Experiments we conducted with TREC data clearly show a significant improvement in the precision of the top ranked documents.

Component Ranking and Automatic Query Refinement for XML Retrieval

“…The AQR algorithm we used is described in [2]. The idea there is to add Lexical Affinity (LA) terms to the query where a Lexical Affinity is a pair of terms that appear close to each other in some relevant documents such that exactly one of the terms appears in the query.…”

Adaptation of the concept hierarchy model with search logs for query recommendation on intranets

“…Knowledge of the query's context, which can be a function of the user's knowledge about the domain, previous search queries, etc., is a crucial factor when suggesting terms that help a user find specific information faster. The two main resources available for query recommendation are the document collection (including anchor logs) [28,24,10,18,20,13] and search logs [3,19,12,4,17,6,5,7,8], which can also be used as forms of implicit or explicit feedback to re-rank retrieved documents.…”

“…The approaches that utilise the collection discover relationships between terms using all documents (global) [9,13] or those retrieved as relevant to a query (local) [24,10]. Terms considered as related to the query are then recommended for query refinement or automatically used for query expansion.…”

Predicting the effectiveness of queries and retrieval systems

“…Conversely, queries that are deemed difficult and which thus achieve only a low retrieval effectiveness degrade with the application of AQE. The rationale is the following: easy queries will have relevant documents among the top ranked results, and therefore an AQE algorithm [29,107,162,163], which derives additional query terms from the top ranked documents returned for the initial query, is likely to derive terms related to the information need 1 . The ranked list retrieved for the expanded query further improves the quality of the results.…”

“…As reason for the degradation in performance of the best queries is given that a highly effective query is actually diluted if additional unnecessary query terms are added to it and the result quality suffers. Carmel et al [29] and Xu and Croft [162] on the other hand only report the worst performing queries to be hurt by the application of AQE.…”