No search result left behind

Huang, Jeff; Lin, Thomas; White, Ryen W.

doi:10.1145/2124295.2124322

Cited by 21 publications

(2 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other proposed online metrics also consider the effect of tabbed browsing (opening several results in browser tabs in quick succession) [88]. Conversely, the lack of a click (abandonment) is often taken as a signal of a lack of relevance, but this interpretation has posed a challenge for evaluating richer search engine result pages, where relevant information may be presented directly, without the need to click.…”

Section: A/b Testingmentioning

confidence: 99%

Search Engines that Learn from Their Users

Schuth

2016

SIGIR Forum

View full text Add to dashboard Cite

More than half the world's population uses web search engines, resulting in over half a billion queries every single day. For many people, web search engines such as Baidu, Bing, Google, and Yandex are among the first resources they go to when a question arises. Moreover, for many search engines have become the most trusted route to information, more so even than traditional media such as newspapers, news websites or news channels on television. What web search engines present people with greatly influences what they believe to be true and consequently it influences their thoughts, opinions, decisions, and the actions they take. With this in mind two things are important, from an information retrieval research perspective. First, it is important to understand how well search engines (rankers) perform and secondly this knowledge should be used to improve them. This thesis is about these two topics: evaluation of search engines and learning search engines. In the first part of this thesis we investigate how user interactions with search engines can be used to evaluate search engines. In particular, we introduce a new online evaluation paradigm called multileaving that extends upon interleaving. With multileaving, many rankers can be compared at once by combining document lists from these rankers into a single result list and attributing user interactions with this list to the rankers. Then we investigate the relation between A/B testing and interleaved comparison methods. Both studies lead to much higher sensitivity of the evaluation methods, meaning that fewer user interactions are required to arrive at reliable conclusions. This has the important implication that fewer users need to be exposed to the results from possibly inferior search engines. In the second part of this thesis we turn to online learning to rank. We learn from the evaluation methods introduced and extended upon in the first part. We learn the parameters of base rankers based on user interactions. Then we use the multileaving methods as feedback in our learning method, leading to much faster convergence than existing methods. Again, the important implication is that fewer users need to be exposed to possibly inferior search engines as they adapt more quickly to changes in user preferences. The last part of this thesis is of a different nature than the earlier two parts. As opposed to the earlier chapters, we no longer study algorithms. Progress in information retrieval research has always been driven by a combination of algorithms, shared resources, and evaluation. In the last part we focus on the latter two. We introduce a new shared resource and a new evaluation paradigm. Firstly, we propose Lerot. Lerot is an online evaluation framework that allows us to simulate users interacting with a search engine. Our implementation has been released as open source software and is currently being used by researchers around the world. Secondly we introduce OpenSearch, a new evaluation paradigm involving real users of real search engines. We describe an implementation of this paradigm that has already been widely adopted by the research community through challenges at CLEF and TREC.

show abstract

Section: A/b Testingmentioning

confidence: 99%

Search Engines that Learn from Their Users

Schuth

2016

SIGIR Forum

View full text Add to dashboard Cite

show abstract

“…The referrer url is used to track consecutive streams of events in the toolbar logs. Contrary to the standard search logs, tracing a linear set of events is not trivial on the toolbar logs given that the user can have several browsing tabs open, and potentially carry unrelated search activity on each one, which are registered in the logs as interleaved events [Huang et al 2012].…”

Section: Toolbar Data Collection and Preparationmentioning

confidence: 99%

Analysis of Search and Browsing Behavior of Young Users on the Web

2014

View full text Add to dashboard Cite

The Internet is increasingly used by young children for all kinds of purposes. Nonetheless, there are not many resources especially designed for children on the Internet and most of the content online is designed for grown-up users. This situation is problematic if we consider the large differences between young users and adults since their topic interests, computer skills, and language capabilities evolve rapidly during childhood. There is little research aimed at exploring and measuring the difficulties that children encounter on the Internet when searching for information and browsing for content. In the first part of this work, we employed query logs from a commercial search engine to quantify the difficulties children of different ages encounter on the Internet and to characterize the topics that they search for. We employed query metrics (e.g., the fraction of queries posed in natural language), session metrics (e.g., the fraction of abandoned sessions), and click activity (e.g., the fraction of ad clicks). The search logs were also used to retrace stages of child development. Concretely, we looked for changes in interests (e.g., the distribution of topics searched) and language development (e.g., the readability of the content accessed and the vocabulary size). In the second part of this work, we employed toolbar logs from a commercial search engine to characterize the browsing behavior of young users, particularly to understand the activities on the Internet that trigger search. We quantified the proportion of browsing and search activity in the toolbar sessions and we estimated the likelihood of a user to carry out search on the Web vertical and multimedia verticals (i.e., videos and images) given that the previous event is another search event or a browsing event. We observed that these metrics clearly demonstrate an increased level of confusion and unsuccessful search sessions among children. We also found a clear relation between the reading level of the clicked pages and characteristics of the users such as age and educational attainment. In terms of browsing behavior, children were found to start their activities on the Internet with a search engine (instead of directly browsing content) more often than adults. We also observed a significantly larger amount of browsing activity for the case of teenager users. Interestingly we also found that if children visit knowledge-related Web sites (i.e., information-dense pages such as Wikipedia articles), they subsequently do more Web searches than adults. Additionally, children and especially teenagers were found to have a greater tendency to engage in multimedia search, which calls to improve the aggregation of multimedia results into the current search result pages.

show abstract

Customizing Websites Through Automatic Web Search

Aldalur

Pérez

Larrinaga

2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The WorldWide Web has endured an incredible growth in the last decades. Nowadays, we can visit an unimaginable number of websites from different devices (laptops, mobiles, tablets...) in order to obtain immediate information. However, this information is separated in different resources and Web information search is unpleasant. The feeling of the users is frustrating when collecting information from different resources. Techniques such as Web personalization and Web customization have been an important research area during the last years. Web customization techniques have been widely used to develop website adaptation along the WorldWide Web. This customization is frequently performed by end-users that use the numerous tools available to carry out this assignment. This article presents Excore, a Web customization tool that permits endusers to customize their websites with automatic Web searches. The article presents the benefits introduced by Excore as a response to the drawbacks end-users experience while they perform their Web activities. Evaluation of the Web customization tool is also addressed in the paper. Evaluation is performed with stakeholders by means of tests and surveys. Results show that testers overcome the detected drawbacks with the use of Excore.

show abstract

No search result left behind

Cited by 21 publications

References 24 publications

Search Engines that Learn from Their Users

Search Engines that Learn from Their Users

Analysis of Search and Browsing Behavior of Young Users on the Web

Customizing Websites Through Automatic Web Search

Contact Info

Product

Resources

About