Search Snippet Evaluation at Yandex: Lessons Learned and Future Directions

Savenkov, Denis; Braslavski, Pavel; Lebedev, Mikhail

doi:10.1007/978-3-642-23708-9_4

Cited by 21 publications

(13 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These results indicate that the participants spent relatively less effort in evaluating the search results in the controversial topic task while addressing more attention to the actual selected webpages. The viewing and result selection patterns were in line with what was reported by previous studies (Balatsoukas & Ruthven, ; Höchstötter & Lewandowski, ; Pan et al, ; Savenkov, Braslavski, & Lebedev, ): The top search results were viewed and selected more often than the ones at the bottom of the SERPs. In addition, the participants verbalized especially the formulation of the search query and did not verbalize thoughts on the evaluation of search result very often.…”

Section: Discussionsupporting

confidence: 89%

Effects of task complexity on online search behavior of adolescents

Walhout

Oomen

Jarodzka

et al. 2017

Asso for Info Science & Tech

View full text Add to dashboard Cite

Evaluation of information during information problem solving processes already starts when trying to select the appropriate search result on a search engine results page (SERP). Up to now, research has mainly focused on the evaluation of webpages while the evaluation of SERPs received less attention. Furthermore, task complexity is often not taken into account. A within subjects design was used to study the influence of task complexity on search query formulation, evaluation of search results and task performance. Three search tasks were used: a fact-finding, cause-effect, and a controversial topic task. To measure perceptual search processes, we used a combination of log files, eye-tracking data, answer forms and think aloud protocols. Results reveal that an increase in task complexity results in more search queries and used keywords, more time to formulate search queries and more considered search results on the SERPs. Furthermore, higher ranked search results were considered more often than lower ranked results. However, not all the results for the most complex task were in line with the expectations. These conflicting results can be explained by a lack of prior knowledge and the possible interference of prior attitudes.Keywords: web search, eye tracking, logging, verbal data, task complexity Cite as: Walhout, J.,Oomen, P., Jarodzka, H., & Brand-Gruwel, S. (2017). Effects of task complexity on online search behavior of adolescents.

show abstract

Section: Discussionsupporting

confidence: 89%

Effects of task complexity on online search behavior of adolescents

Walhout

Oomen

Jarodzka

et al. 2017

Asso for Info Science & Tech

View full text Add to dashboard Cite

show abstract

“…Recent work has examined the generation of snippets from more complex angles -from manipulating underlying indexes [4,49] to language modelling [14,32], as well as using user search data to improve the generation process [1,42]. Previous generation approaches also may not consider what parts of a document searchers actually nd useful.…”

Section: Generating Snippet Textmentioning

confidence: 99%

A Study of Snippet Length and Informativeness

Maxwell

Azzopardi

Moshfeghi

2017

Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

e design and presentation of a Search Engine Results Page (SERP) has been subject to much research. With many contemporary aspects of the SERP now under scrutiny, work still remains in investigating more traditional SERP components, such as the result summary. Prior studies have examined a variety of di erent aspects of result summaries, but in this paper we investigate the in uence of result summary length on search behaviour, performance and user experience. To this end, we designed and conducted a withinsubjects experiment using the TREC AQUAINT news collection with 53 participants. Using Kullback-Leibler distance as a measure of information gain, we examined result summaries of di erent lengths and selected four conditions where the change in information gain was the greatest: (i) title only; (ii) title plus one snippet; (iii) title plus two snippets; and (iv) title plus four snippets. Findings show that participants broadly preferred longer result summaries, as they were perceived to be more informative. However, their performance in terms of correctly identifying relevant documents was similar across all four conditions. Furthermore, while the participants felt that longer summaries were more informative, empirical observations suggest otherwise; while participants were more likely to click on relevant items given longer summaries, they also were more likely to click on non-relevant items. is shows that longer is not necessarily be er, though participants perceived that to be the case -and second, they reveal a positive relationship between the length and informativeness of summaries and their a ractiveness (i.e. clickthrough rates). ese ndings show that there are tensions between perception and performance when designing result summaries that need to be taken into account.

show abstract

“…With respect to the effect of domain knowledge on eye gaze, it was found that users with a high level of domain knowledge are less likely to attend to the element of title, as demonstrated by this study's undergraduate and postgraduate biology users with a high level of biology knowledge. However, a previous finding has suggested that titles of lower ranked items receive more attention than the snippets of higher ranked items (Savenkov et al, 2011), and the title and abstract of SERPs receive more attention than other elements by general public users (Wang et al, 2015). As such, this study has provided evidence that a user's level of domain knowledge affects eye gaze behaviour.…”

Section: Areas Of Interestmentioning

confidence: 56%

“…In Web search environments, users paid more attention to the top three, next top three and top advertisements than other regions, such as related searches on search engine results page (SERP) (Dumais et al, 2010), while few abstracts in SERPs from Google and Yahoo were viewed in query reformulation (Lorigo et al, 2008). However, the title of lower ranked items was considered more important than the snippets of higher ranked items (Savenkov, Braslavski, & Lebedev, 2011). Wang, Xie, and Lee's (2015) study found that the title and abstract of SERPs were ranked the top two elements in terms of total fixation time and fixation count.…”

Section: Eye Gaze and Search Interface Designmentioning

confidence: 99%

Natural Search User Interfaces for Complex Biomedical Search: An Eye Tracking Study

Liu

Thomas

Bacic

et al. 2017

Journal of the Australian Library and Information Association

View full text Add to dashboard Cite

Controlled vocabularies such as Medical Subject Headings (MeSH) have been extensively used to organise information resources in the biomedical domain. However, the usefulness of these terms for information access has not been rigorously evaluated in interactive search environments. The objective of this study was to gain an understanding of domain experts' interactions with novel search interfaces within the context of biomedical information search, with a goal of better interface design of information retrieval systems. An eye tracking study of biomedical domain experts' interactions with novel search interfaces was conducted, considering user's individual differences and gaze behaviour. The findings suggest that types of search interfaces have significant effects on gaze behaviour in terms of fixation-based measures of areas of interest, i.e. visual attention to the elements of title, author, abstract and MeSH terms in document surrogates. Significant interaction effects between cognitive style and user interaction with search interfaces were found, specifically the amount of attention to MeSH terms by analytic and wholistic searchers. These findings contribute to our understanding of the relationship between cognitive styles and gaze patterns in information search. Based on these findings, the implications of individual differences and gaze behaviour for search interface design are discussed.

show abstract

Search Snippet Evaluation at Yandex: Lessons Learned and Future Directions

Cited by 21 publications

References 13 publications

Effects of task complexity on online search behavior of adolescents

Effects of task complexity on online search behavior of adolescents

A Study of Snippet Length and Informativeness

Natural Search User Interfaces for Complex Biomedical Search: An Eye Tracking Study

Contact Info

Product

Resources

About