Lessons learned: usability testing a federated search product

Abstract

PurposeUsability testing using think aloud protocols was conducted on the MetaLib interface, a new federated search product soon to be offered by the university libraries. The object of the testing was to find problems users might experience with the MetaLib interface in order to improve site usability.Design/methodology/approachBased on responses to a demographic questionnaire, a representative sample of eight volunteers, diverse with respect to affiliation, discipline, gender, language, and computer expertise was selected. Using the MetaLib interface and guided by a moderator, participants completed real‐world tasks while verbalizing their thoughts. Participants also completed a questionnaire and answered three open‐ended items. Transcripts provided the data for the study, which were used to determine problems and difficulties with the interface.FindingsThe usability testing revealed one critical issue, that is, problems with the login. Other issues included problems with primary and secondary navigation, confusing terminology, and inconsistency with the site design and user expectations.Research limitations/implicationsThis was a qualitative study and results should not be used to generalize to a larger population, but rather to identify issues with usability.Practical implicationsThe issues identified will provide feedback that will be used to address the usability of the software and the design of training, tutorials, and quick guides.Originality/valueResults of this study will add to the body of information about the usability of a federated search product and can help others to select and/or design a usable product. This study can also be used for the future improvement of the MetaLib product.

Graduate Students and Federated Searching

“…After the implementation process was over, many academic librarians then conducted usability studies to resolve problems with interface design and navigation, and to design tutorials (Cervone, 2005, Tallent, 2004Wrubel & Schmidt, 2007;George, 2008).…”

Students and Federated Searching

“…12 Users tend to experience difficulty with navigating results pages and confusion with library-specific terminology and icons. 13 Because of this confusion, Elliot concludes that students also have trouble accessing full-text content from a federated results page.…”

One box to search them all

“…Her summary of the situation was that "[w]hile libraries are indeed conducting usability testing and other types of user studies, it is clear that librarian preferences rather than user preferences still rule the design of library-oriented metasearch tools" (Elliott, 2004, p. 23). The literature shows that extensive usability testing is performed in most cases (George, 2008;Avery, Ward and Janicke Hinchliffe, 2007;Boock, Nichols and Kristick, 2006). Avery (2007) reports spending 15 weeks on just usability testing and tweaking the interface.…”