Training for Web search: Will it get you in shape?

Abstract

. Two of the experimental treatments were based on the type of training the users received. The first of these treatments was the presence or absence of training on the principles of the use of Boolean operators, henceforth referred to as logic training. The second was the presence or absence of training that focused on the specific characteristics of the assisted search interface, which we will refer to as interface training. The third treatment was the type of interface used by the participants, namely, simple or assisted. The impacts of logic and interface training and the type of search interface used on correctness, time, satisfaction, and confidence were measured and analyzed. Given that time is money, Web searching can be a very expensive proposition. Even with the best search technology, the usefulness of search results depends on the searcher's ability to use that technology effectively. In an effort to improve this ability, our research investigates the effects of logic training, interface training, and the type of search interface on the search process. In a study with 145 participants, we found that even limited training in basic Boolean logic improved performance with a simple search interface. Surprisingly, for users of an interface that assisted them in forming syntactically correct Boolean queries, performance was negatively affected by logic training and unaffected by interface training. Use of the assisted interface itself, however, resulted in strong improvements in performance over use of the simple interface. In addition to being useful for search engine providers, these findings are important for all companies that rely heavily on search for critical aspects of their operations, in that they demonstrate simple means by which the search experience can be improved for their employees and customers.

In the bookshop

“…Researchers have repeatedly noted or criticized the simple strategies adopted by many users [16,22]. These criticisms have often identified particular patterns of simplistic searching: limited number of query terms (words); minimal use of search modifiers (phrase search, compulsory terms); poor understanding of Boolean and logical techniques..…”

Toward a Learning Science for Complex Crowdsourcing Tasks

“…Several controlled experiments have shown the efficacy of various web search training interventions. Lucas and Topi showed that training users in Boolean logic helped them achieve higher accuracy in their later searches [18]. Harvey et al showed that presenting crowdworkers with query suggestions that were more effective than their own enabled them to later generate higher quality queries [10].…”

Learning and Training to Search

“…The findings strongly support the value of providing a novice searcher with both task and search strategy validity feedback. All of the experiments reported in Debowski's (2002) study testify about the need to develop effective search training approaches; even after training, the performance level in these experiments was quite low, as it often is with complex search tasks (see, for example, Sutcliffe, Ennis, and Watkinson (2000) and Lucas and Topi (2004)). Debowski (2002) also points out the importance of integrating training and ongoing support.…”

“…After taking action to execute the query, the searcher must then know how to interpret the results, which is based on system-specific information, such as ordering of results by relevancy; computer usage familiarity, such as using a find feature to search within documents; human intelligence, and other personal characteristics. Figure 2, which first appeared in (Lucas and Topi, 2004), models the learning domain by those human-dependent factors that ultimately affect the success of a search. Search success is defined here, as per Chan, Tan, and Wei (1999) and Te'eni and Feldman (2001), in terms of search performance (correctness/accuracy, time) and searchers' attitudes (confidence and satisfaction).…”

“…This carefully designed study with a large sample suggests that the workshop was effective in improving the participants' skills in question formulation, raising confidence in their ability to search, developing search skills, and improving search outcomes. Lucas and Topi (2004) and Topi and Lucas (2005) evaluated the effectiveness of brief computer-based training interventions on the searcher's ability to retrieve correct answers from a simulated intranet site with a commonly used search engine. Their results suggest that even a modest Boolean training intervention can be very effective in cases where the user interface does not provide the support that the users' expertise level would warrant.…”