Evaluating implicit measures to improve web search

Abstract

Of growing interest in the area of improving the search experience is the collection of implicit user behavior measures (implicit measures) as indications of user interest and user satisfaction. Rather than having to submit explicit user feedback, which can be costly in time and resources and alter the pattern of use within the search experience, some research has explored the collection of implicit measures as an efficient and useful alternative to collecting explicit measure of interest from users.This research article describes a recent study with two main objectives. The first was to test whether there is an association between explicit ratings of user satisfaction and implicit measures of user interest. The second was to understand what implicit measures were most strongly associated with user satisfaction. The domain of interest was Web search. We developed an instrumented browser to collect a variety of measures of user activity and also to ask for explicit judgments of the relevance of individual pages visited and entire search sessions. The data was collected in a workplace setting to improve the generalizability of the results.Results were analyzed using traditional methods (e.g., Bayesian modeling and decision trees) as well as a new usage behavior pattern analysis (“gene analysis”). We found that there was an association between implicit measures of user activity and the user's explicit satisfaction ratings. The best models for individual pages combined clickthrough, time spent on the search result page, and how a user exited a result or ended a search session (exit type/end action). Behavioral patterns (through the gene analysis) can also be used to predict user satisfaction for search sessions.

MyWebSteps: Aiding Revisiting with a Visual Web History

“…Around 50% of webpages are looked at for 12 seconds or less, 70% for 30 seconds or less, and only 10% for more than two minutes (Weinreich et al, 2006). A dwell time of 30 seconds or more on a webpage can be indicative of webpage utility (Fox et al, 2005), and this threshold was used to analyze search trails in Web logs (White and Huang, 2010). The present study used this threshold to select webpages from participants' Web history for the revisiting experiment.…”

The Design of a Visual History Tool to Help Users Refind Information within a Website

“…Dwell time on a webpage is also an important factor for revisiting because previous research [12,30] has shown that, during search and navigation activities, a dwell time of 30 seconds is indicative of a page that is of interest to a participant (i.e., informational rather than just navigational). A three month weblog revealed that more than two thirds of visits were navigational based on that criteria [2], so using this feature as a filter could substantially reduce the amount of history information that needs to be considered during revisiting.…”

“…Interest: The time which users spend on a page reveals indicates the extent to which they are interested in it. Previous research [12,30] has shown that, during search and navigation activities, a dwell time of 30 seconds is indicative of a page that is of interest to a participant (i.e., informational rather than just navigational). Another study [2] reported that only 31% of visited webpages were informational and, when people tried to revisit a webpage which had not been visited recently or frequently, that webpage often was informational.…”

Complex adaptive filtering user profile using graphical models

“…Choosing the graphical modelling approach as a unified framework to combine multiple forms of evidence is motivated by the prior research. Recently, there has been some independent work using a different graphical modelling approach (dependency networks) to discover the relationships between implicit measures and explicit satisfaction, and using decision tree for prediction (Fox et al, 2005). They were focusing on predicting user satisfaction with web search results based on implicit measures gathered while users were conducting their searches and viewing results.…”