Stuck in traffic

Abstract

In this paper we investigate how query response delays and document download delays a↵ect user interactions within a search system. Guided by Information Foraging Theory and Search Economic Theory, five competing hypotheses relating to the behaviours of searchers in the presence of delays are considered and examined in the context of ad-hoc topic retrieval. A between-subjects laboratory study with 48 undergraduate subjects was conducted. Subjects were randomly assigned to one of four conditions that varied the type of delay experienced. When faced with query response delays, subjects did not examine more documents per query as expected. However, when the total amount of time spent per query (a combination of delay and querying time) increased, subjects did examine more documents per query. When faced with document download delays, subjects did not spent more time within documents. Subjects however did spend longer within documents when subjected to both query and document delays. We found a strong and significant correlation between query time (independent of delay) and the interactions of subjects in terms of the number of queries posed, the number of documents examined, and the depth to which subjects went. These findings contrast with previous works on how delays a↵ect search behaviour, and suggest that the theory needs to be refined to make more credible predictions relating to search behaviours.

A Systematic Review of Cost, Effort, and Load Research in Information Search and Retrieval, 1972–2020

“…For example, Azzopardi et al [8] found that users searching on the structured interface (considered high cost) issued significantly less queries and examined significantly more documents per query than those using the lower cost interfaces. Maxwell and Azzopardi [96] observed a similar finding, with their results indicating that as the relative cost of querying increased, users issued fewer queries and examined more documents per query. As the information search task becomes more complex, users perceive higher levels of effort [91] and workload [120], and effort measures have been found to correlate with all information task characteristics associated with task difficulty (i.e., higher number of steps to achieve the task goal; intellectual task product; amorphous task goal).…”

“…A study examining the effects of time delays on the Google search engine found that even a relatively small delay of 400ms in returning search results led to a reduction in the number of searches conducted by 0.59% over the course of 6-weeks [16]. Additional studies revealed similar findings, highlighting that as the cost of querying increases, the number of queries issued by the user will decrease [8,96].…”

“…For example, "cost is often measured by the time of a series of actions, such as formulating queries, examining snippets, clicks on results etc" [91]; and "cost is often considered as the amount of time spent" [9]. • Interaction Orientated/Count Based: Cost was also defined in relation to the interactions/number of actions occurring between the user and the system (𝑁 = 4) [9,22,96,144]. For example, "consider the cost of information search results from query generation, documentation examination, search engine result pages and task description examinations" [144].…”

Untangling Cost, Effort, and Load in Information Seeking and Retrieval

“…Search interaction data was used in the measurement of all three CEL concepts with often the same metrics used to measure different concepts. For example, "number of queries issued", "number of documents opened", and "number of pages viewed" have been implicated as both a measure of cost [33,43] and effort [8,21,27]. Likewise, "time-on-task" was used as a metric for all CEL concepts: cost [43,62]; effort [8,48,54]; and load [13,46], implying that time-on-task is an adequate indicator of all three concepts.…”

“…Search interaction logs cover a variety of different actions ranging from a simple mouse click to viewing a whole document. As observed from the studies [4,8,33,44,62], these actions are often "count based". However, it is not clear how we can compare, say the total number of clicks to the total number of queries examined?…”