A Usability Evaluation of Academic Virtual Reference Services

Abstract

This study examined the usability of five virtual reference services-instant messenger chat, e-mail, telephone, text messaging, and Skype videoconferencing-by having 31 undergraduate and graduate students evaluate the usability of the virtual reference services of two different universities. The study's results suggest that user preference and satisfaction for virtual reference service are highly correlated with the service's overall usability in terms of effectiveness and efficiency. Online chat was rated highest across all measures including satisfaction and seven different usability factors. Major implications of the study suggest that online chat is the virtual reference of choice for university students and that usability metrics are a good predictor of user preferences centered on high return on investment, speed of transaction, convenience, and minimal effort.o stay relevant in this rapidly evolving information age, libraries across the world are racing to reinvent themselves. They must do so in an environment where information seekers have "…many options, little patience, and use many different types of information and communication technology."1 While many libraries are increasingly making their content available digitally over the Internet, they are also taking steps to make their reference services available virtually. Some virtual reference services, including email, instant messaging (online chat), and telephone have been around for decades, while other services such as text messaging and videoconferencing are just beginning to emerge. Sloan reported that e-mail reference has been going strong for 20 years and live chat-based reference for 10 years. 2Virtual reference services are becoming commonplace in both academic and public libraries across the United States. In a recent survey study of 1,226 academic and public libraries, 54.95 percent of survey respondents indicated they offer reference services virtually. In a separate phase of this study, the researchers found that 84.6 percent (n = 203) of library websites evaluated offered virtual reference services.3 The most common type of virtual reference was e-mail reference, followed by chat, then text, then "other." Both surveys found the least used form of virtual reference was Skype or video conferencing.While many libraries now offer a suite of virtual reference services, the proliferation of these services is not always matched by actual use. 4 Many virtual crl13-408

“Ask a librarian”: Comparing virtual reference services in an Israeli academic library

“…Steiner (2011) writes that physical reference service remains the best option for those who feel their questions are too nebulous for quick answers or who prefer the familiarity of working with a known and trusted librarian. Participants in the study of Chow and Croxton (2014) appeared to prefer the specific type of reference medium that most conveniently met their needs at any given time. Connaway and Radford (2011) suggested that VRS allows libraries to offer an alternative that may be important to students who find faceto-face interactions intimidating.…”

“…Chow and Croxton (2014) found that online chat was rated highest in effectiveness, efficiency, and satisfaction among VR services. One reason for this was because transactions were faster using chat in comparison to all other services.…”

Delivering Virtual Reference Services on the Web: An Investigation into the Current Practice by Academic Libraries

“…Surveys analyze usage of increasing numbers of technologies including email and chat (Dee & Allen, 2006); email, chat, and phone (Mon et al, 2008); email-only reference, synchronous reference, and no virtual reference (Mu et al, 2011);email, chat, and text (Dorris et al, 2009); email, reference forms, forums, video conferencing, and chat (Bao, 2003); and finally email, chat, text, and video conferencing (Chow & Croxton, 2014). Clearly virtual reference services are popular but none of the studies above are conducted on a large scale.…”

“…These studies ask how users discover reference services on libraries' websites with particular attention to the placement of the link to services on libraries' websites (Chow & Croxton, 2014;Dee & Allen, 2006;Mu, Dimitroff, Jordan, & Burclaff, 2011). Bao (2003) reports only 52% of the libraries in the sample presented web-based interactive reference services on their homepages.…”

Do You Want to Chat? Reevaluating Organization of Virtual Reference Service at an Academic Library

“…that online chat reference is suitable mostly for simple factual and directional but not reference questions." 11 Cabaniss's analysis discovered that at the University of Washington Libraries, the majority of questions consisted of general information and known-item searches, queries that could be answered by graduate student assistants. 12 However, other studies mention the extent to which instruction is taking place within the chat environment, suggesting that, in many cases, the service moves beyond simply answering factual questions and provides an experience to users that allows them to develop new skills.…”