Advantages of query biased summaries in information retrieval

Abstract. Presentation of search results in Web-based information retrieval (IR) systems has been dominated by a textual form of information such as the title, snippet, URL, and/or file type of retrieved documents. On the other hand, document's visual aspects such as the layout, colour scheme, or presence of images have been studied in a limited context with regard to their effectiveness of search result presentation. This paper presents a comparative evaluation of textual and visual forms of document summaries as the additional document surrogate in the search result presentation. In our study, a sentence-based summarisation technique was used to create a textual document summary, and the thumbnail image of web pages was used to represent a visual summary. The experimental results suggest that both have the cases where the additional elements contributed to a positive effect not only in users' relevance assessment but also in query re/formulation. The results also suggest that the two forms of document summary are likely to have different contexts to facilitate user's search experience. Therefore, our study calls for further research on adaptive models of IR systems to make use of their advantages in appropriate contexts.

Section: Background and Motivationmentioning

confidence: 99%

A Comparative Study of the Effectiveness of Search Result Presentation on the Web

Joho

Jose

2006

“…Subjects were asked to identify (mark) as many relevant documents as they could over four topics, with each subject allocated a total of 10 minutes per topic 6 . For each topic, the search system was configured to present query biased snippets [35] of different lengths. For this study however, we consider only one of those interfaces -where two fragments were presented.…”

Section: User Study Subjects Costs and Probabilitiesmentioning

confidence: 99%

Information Scent, Searching and Stopping

Maxwell

Azzopardi

2018

Abstract. Current models and measures of the Interactive Information Retrieval (IIR) process typically assume that a searcher will always examine the first snippet in a given Search Engine Results Page (SERP), and then with some probability or cutoff, he or she will stop examining snippets and/or documents in the ranked list (snippet level stopping). Prior work has however shown that searchers will form an initial impression of the SERP, and will often abandon a page without clicking on or inspecting in detail any snippets or documents. That is, the information scent affects their decision to continue. In this work, we examine whether considering the information scent of a page leads to better predictions of stopping behaviour. In a simulated analysis, grounded with data from a prior user study, we show that introducing a SERP level stopping strategy can improve the performance attained by simulated users, resulting in an increase in gain across most snippet level stopping strategies. When compared to actual search and stopping behaviour, incorporating SERP level stopping offers a closer approximation than without. These findings show that models and measures that naïvely assume snippets and documents in a ranked list are actually examined in detail are less accurate, and that modelling SERP level stopping is required to create more realistic models of the search process.

“…Evidence suggests that in some cases a document as a whole might not be relevant to the query, but a subtopic of it may be highly relevant [1], summarization improves accuracy and speed of user relevance judgments [2], and even relevant documents may act as "poison pills" and harm topic precision after feedback [3], This problem is compounded in Blind Relevance Feedback (BRF) where all top ranked documents from an initial retrieval run are assumed to be relevant, but often one or more of them will not be. In this case, terms not related to the topic of the query, but which meet the selection criteria are used for QE (e.g.…”

Section: Introductionmentioning

confidence: 99%

Query Expansion for Language Modeling Using Sentence Similarities

Ganguly

Leveling

Jones

2011

Abstract. We propose a novel method of query expansion for Language Modeling (LM) in Information Retrieval (IR) based on the similarity of the query with sentences in the top ranked documents from an initial retrieval run. In justification of our approach, we argue that the terms in the expanded query obtained by the proposed method roughly follow a Dirichlet distribution which, being the conjugate prior of the multinomial distribution used in the LM retrieval model, helps the feedback step. IR experiments on the TREC ad-hoc retrieval test collections using the sentence based query expansion (SBQE) show a significant increase in Mean Average Precision (MAP) compared to baselines obtained using standard term-based query expansion using LM selection score and the Relevance Model (RLM). The proposed approach to query expansion for LM increases the likelihood of generation of the pseudo-relevant documents by adding sentences with maximum term overlap with the query sentences for each top ranked pseudorelevant document thus making the query look more like these documents. A per topic analysis shows that the new method hurts less queries compared to the baseline feedback methods, and improves average precision (AP) over a broad range of queries ranging from easy to difficult in terms of the initial retrieval AP. We also show that the new method is able to add a higher number of good feedback terms (the golden standard of good terms being the set of terms added by True Relevance Feedback). Additional experiments on the challenging search topics of the TREC-2004 Robust track show that the new method is able to improve MAP by 5.7% without the use of external resources and query hardness prediction typically used for these topics.