Using off-the-shelf search technology provides a single point of access into library resources, but we found that such commercial systems are not entirely satisfactory for the academic library setting. In response to this, Oregon State University (OSU) Libraries designed and deployed LibraryFind, a metasearch system. We conducted a usability experiment comparing LibraryFind, the OSU Libraries Web site, and Google Scholar. Each participant used all three search systems in a controlled setting, and we recorded their behavior to determine the effectiveness and efficiency of each search system. In this article, we focus on understanding what factors are important to undergraduates in choosing their primary academic search system for class assignments. Based on a qualitative and quantitative analysis of the results, we found that mimicking commercial Web search engines is an important factor to attract undergraduates; however, when undergraduates use these kinds of search engines, they expect similar performance to Web search engines, including factors such as relevance, speed, and the availability of a spell checker. They also expected to be able to find out what kinds of content and materials are available in a system. Participants' prior experience using academic search systems also affected their expectations of a new system.
IntroductionToday's academic library Web sites serve as gateways to digitally accessible library resources, ranging from full-text newspaper and journal articles to online catalogs of physical collections. Yet, undergraduate students often use Web-based search engines as primary tools to find class-related information not only because they are familiar with Web-based search engines (Augustine & Greene, 2002;Notovny, 2004) but also because they struggle to use library Web sites as a gateway for finding information (Chisman, Diller, & Walbridge, 1999; Eliason, McKinstry, & Fraser, 1997;Notovny, 2004;Stephan, Cheng, & Young, 2006;Yakel, 2004).Many researchers in library and information science focus on improving library Web sites so that navigating library services is easy. These include local resources (e.g., the library catalog) as well as databases offered by third-party vendors (e.g., Academic Search Premier). Yet, navigation remains difficult for novice users unfamiliar with exploiting the multiplicity of library services: They do not know where to start in part because they do not know which databases are appropriate for their current information need (Eliason et al., 1997;Stephan et al., 2006). Novice users also find it difficult to identify useful documents from the returned results (Eliason et al., 1997;Stephan et al., 2006). Undergraduates expect disparate library resources to be aggregated so that a single query will return satisfactory results (Augustine & Greene, 2002;Notovny, 2004).One promising approach to these persistent issues is simplifying the search rather than constantly revising the library Web site to improve navigation. For that reason, researchers are experimenting wi...