Real life, real users, and real needs: a study and analysis of user queries on the web

Abstract. There is overwhelming evidence suggesting that the real users of IR systems often prefer using extremely short queries (one or two individual words) but they try out several queries if needed. Such behavior is fundamentally different from the process modeled in the traditional test collection-based IR evaluation based on using more verbose queries and only one query per topic. In the present paper, we propose an extension to the test collection-based evaluation. We will utilize sequences of short queries based on empirically grounded but idealized session strategies. We employ TREC data and have test persons to suggest search words, while simulating sessions based on the idealized strategies for repeatability and control. The experimental results show that, surprisingly, web-like very short queries (including one-word query sequences) typically lead to good enough results even in a TREC type test collection. This finding motivates the observed real user behavior: as few very simple attempts normally lead to good enough results, there is no need to pay more effort. We conclude by discussing the consequences of our finding for IR evaluation.

show abstract

“…Real-life searchers often prefer very short queries [1,4]. They may also avoid excessive browsing [1,9].…”

Section: User Behaviormentioning

confidence: 99%

“…Recent studies show that real users of information retrieval (IR) systems search by very short queries but may try out several queries in a session [1][2][3][4][5]. Such queries may consist of only 1-2 search keys.…”

Section: Introductionmentioning

confidence: 99%

Test Collection-Based IR Evaluation Needs Extension toward Sessions – A Case of Extremely Short Queries

Keskustalo

Järvelin

Pirkola

et al. 2009

Information Retrieval Technology

View full text Add to dashboard Cite

show abstract

“…• Advanced search features such as Boolean operators are not used much, and users often make mistakes in using them [375].…”

Section: Transactional Queriesmentioning

confidence: 99%

Workload Modeling for Computer Systems Performance Evaluation

Feitelson

2015

196

131

View full text Add to dashboard Cite

Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

show abstract

“…This presents quite a challenge, especially when search engine queries are rather short. Studies by Spink et al [16] [27] have shown that the average query length is between two and three keywords. Moreover, these requests tend to cover a rather wide variety of information needs, and are often expressed with ambiguous terms.…”

Section: Introductionmentioning

confidence: 99%

Term Proximity Scoring for Keyword-Based Retrieval Systems

Rasolofo

Savoy

2003

Lecture Notes in Computer Science

119

111

View full text Add to dashboard Cite

Abstract. This paper suggests the use of proximity measurement in combination with the Okapi probabilistic model. First, using the Okapi system, our investigation was carried out in a distributed retrieval framework to calculate the same relevance score as that achieved by a single centralized index. Second, by applying a term-proximity scoring heuristic to the top documents returned by a keyword-based system, our aim is to enhance retrieval performance. Our experiments were conducted using the TREC8, TREC9 and TREC10 test collections, and show that the suggested approach is stable and generally tends to improve retrieval effectiveness especially at the top documents retrieved.

show abstract

Real life, real users, and real needs: a study and analysis of user queries on the web

Cited by 1,064 publications

References 12 publications

Test Collection-Based IR Evaluation Needs Extension toward Sessions – A Case of Extremely Short Queries

Test Collection-Based IR Evaluation Needs Extension toward Sessions – A Case of Extremely Short Queries

Workload Modeling for Computer Systems Performance Evaluation

Term Proximity Scoring for Keyword-Based Retrieval Systems

Contact Info

Product

Resources

About