Evaluating advanced search interfaces using established information‐seeking models

Abstract

When users have poorly defined or complex goals, search interfaces offering only keyword searching facilities provide inadequate support to help them reach their information-seeking objectives. The emergence of interfaces with more advanced capabilities such as faceted browsing and result clustering can go some way to some way toward addressing such problems. The evaluation of these interfaces, however, is challenging since they generally offer diverse and versatile search environments that introduce overwhelming amounts of independent variables to user studies; choosing the interface object as the only independent variable in a study would reveal very little about why one design out-performs another. Nonetheless if we could effectively compare these interfaces we would have a way to determine which was best for a given scenario and begin to learn why. In this article we present a formative inspection framework for the evaluation of advanced search interfaces through the quantification of the strengths and weaknesses of the interfaces in supporting user tactics and varying user conditions. This framework combines established models of users, user needs, and user behaviours to achieve this. The framework is applied to evaluate three search interfaces and demonstrates the potential value of this approach to interactive IR evaluation.

Exploration of Applying a Theory-Based User Classification Model to Inform Personalised Content-Based Image Retrieval System Design

“…To tackle this problem, the deductive approach has recently be considered, which is to develop and validate user models based on established user behaviour theories, and then apply the validated user models to search systems. For example, Wilson et al (2009) [13] presented a formative inspection framework based on two established user models, namely the ISS conditions from episodic model and the levels of search strategies in strategic model. The framework was applied to evaluate three search interfaces.…”

“…Many of the existing studies focused on improving the search accuracy and efficiency, while a few of them focused on improving the design of the search system and the users' overall search experiences [1,4,11,13]. All the studies above had evidenced the user models can be generated based on the users' interaction data or established theories or both.…”

“…The user models are the key element to learn different user preferences for different user types [1,4,11], and to design and develop effective personalised search systems [4,13]. Whilst the theory-based user models [1,4,13] have been developed for personalised search, there has been lack of a methodology for systematic empirical studies with real search systems in order to utilise the user models for better understanding the users. Among all of the studies, the information foraging theory based ISE model [4] is particularly developed for CBIR.…”

“…Extensive work has been done to provide personalised search for improving users' overall search effectiveness and experiences [1,4,9,11,13,14,15]. However, there still exist gaps and challenges on various aspects.…”

Cognitive traveling in digital space: from keyword search through exploratory information seeking

“…Both targeted and exploratory search are directly related to the effective visualization of search results overview and then the results themselves (documents, pages, information). Current approaches in the exploratory search, in addition to traditional tabular display of search results, make a significant focus on supporting the process of searching through faceted browsers like mSpace [88], RB or OntoViews [50], on improving the user orientation and understanding of information using link or content annotations. Essential is also the support for creation of visual search queries by using approaches based on views, visualization of search and browsing history [53] with graphs and interactive visualization of relationships between search results using (hierarchical) graphs [67].…”