Task complexity, vertical display and user interaction in aggregated search

Abstract

Aggregated search is the task of blending results from specialized search services or verticals into the Web search results. While many studies have focused on aggregated search techniques, few studies have tried to better understand how users interact with aggregated search results. This study investigates how task complexity and vertical display (the blending of vertical results into the web results) affect the use of vertical content. Twenty-nine subjects completed six search tasks of varying levels of task complexity using two aggregated search interfaces: one that blended vertical results into the web results and one that only provided indirect vertical access. Our results show that more complex tasks required significantly more interaction and that subjects completing these tasks examined more vertical results. While the amount of interaction was the same between interfaces, subjects clicked on more vertical results when these were blended into the web results. Our results also show an interaction between task complexity and vertical display; subjects clicked on more verticals when completing the more complex tasks with the interface that blended vertical results. Subjects' evaluations of the two interfaces were nearly identical, but when analyzed with respect to their interface preferences, we found a positive relationship between system evaluations and individual preferences. Subjects justified their preference using similar rationales and their comments illustrate how the display itself can influence judgments of information quality, especially in cases when the vertical results might not be relevant to the search task. Categories and Subject Descriptors General TermsPerformance, Experimentation, Human Factors. KeywordsAggregated search interfaces, search behaviors, evaluation, user study, interaction, task complexity INTRODUCTIONIn addition to Web search, commercial search companies (e.g., Google, Bing, Yahoo!) provide access to a wide range of specialized services known as verticals (e.g., images, video, news Most published research in aggregated search has focused on automatic methods for predicting which verticals to present (vertical selection) [4,5,11,19] and where in the Web results to present them (vertical presentation) [2,3,23]. Evaluation of these systems has typically been conducted by using editorial vertical relevance judgements as the gold standard [2,3,4,5,19], or by using user-generated clicks on vertical results as a proxy for relevance [11,23]. While these studies have greatly advanced the state of the art in aggregated search techniques, because users are far removed from the evaluation, they have contributed little insight about how users' higher-level objectives influence their engagement with vertical search results.A few published studies have investigated user behavior with aggregated search interfaces [24,25,28]. Thus far, these studies show two major trends. First, when a vertical is relevant, users prefer to see its results towards to the top of the blended results [...

Vertical selection in the information domain of children

“…This hypothesis is also supported by the fact that the best performing verticals at higher threshold values are the image oriented verticals (Google Images and Worksheets). The bias towards visual oriented verticals has also been reported before in the literature [4,20]. Anecdotally we also observed that children oriented educational websites (e.g.…”

Searching as learning: A systematization based on literature

“…Some studies have explored how search formulation and source selection vary by the level (complexity) of cognitive learning processes based on Bloom's taxonomy [34][35][36]. There are also studies that categorize search tasks using characteristics that can be reduced at least to a certain extent to Bloom's taxonomy.…”

The effect of cognitive abilities on information search for tasks of varying levels of complexity

“…Similarly, Bell and Ruthven defined task complexity as a function of the a priori determinability of the required information, the search strategy, and the ability to judge relevance [9]. Finally, Jansen et al [31] (and later Arguello et al [5] and Wu et al [53]) defined task complexity in terms of the amount of cognitive effort and learning required to complete the task. To this end, they adopted a taxonomy of learning outcomes originally proposed by Anderson and Krathwohl [3] for characterizing educational materials.…”