User Evaluation Methods for Visual Web Search Interfaces

Abstract

In recent years, numerous visual Web search interfaces have been developed in the research community. However, the user evaluations of these interfaces have been performed using a wide range of methods, making it difficult to compare and verify the relative value of the proposed advancements. In this paper, we survey these evaluation methods, and propose a stepped evaluation and refinement model for the systematic study and enhancement of visual Web search interfaces. We suggest that this stepped model can be generalized to support the evaluation of other information visualization systems that target exploratory or knowledge-centric domains.

CIDER: Concept-based image diversification, exploration, and retrieval

“…It provides researchers with a means to verify and validate design assumptions, confirm or reject hypotheses, and make comparisons between different systems and techniques (Hoeber, 2009). Evaluating Web search interfaces is challenging, mainly due to the knowledge-centric nature of the domain.…”

“…The issue of evaluation is further complicated when the Web search interface includes highly visual and interactive elements. Methods for evaluating information visualization systems have recently become a rather active research topic (Carpendale, 2008;Hoeber, 2009).…”

“…The amount of knowledge the participants are gaining as they perform their assigned search activities is very difficult to estimate (Hoeber, 2009). While it may be possible to quiz the participants at the end of the session, separating the ability of the participant to learn about the topic from the support the search interface provides may not be possible.…”

“…A more realistic evaluation could allow for the participants to choose their own image search tasks, and to conduct their searches over an extended period of time. Studying CIDER in such a longitudinal setting may provide valuable insights regarding the learnability, usability, and utility of the interface by capturing specific search and interaction operations, along with the participants ongoing impressions of the system during realworld use as they become experts in the features of the software (Hoeber, 2009).…”

Automatic Topic Learning for Personalized Re-Ordering of Web Search Results

“…In addition, user evaluations are in the planning stages, which will allow us to determine the willingness of searchers to pre-select topics during their search process. A longitudinal study will allow us to evaluate the value of the personalization methods in real-world search settings [6].…”