Development and Testing of a Reference Transaction Assessment Instrument

Abstract

An i~J!Jas designed to assess the reference outcome in terms of patron report of success. It was theorized that patrons were capable of separating reports of success in finding what was wanted from reports of general overall satisfaction, if given a suitably constructed instrument. Utilized in design were Cronbach 's theories on response set and Cuadra's theories on the fail-safe reaction. A survey was conducted. Fifteen academic libraries across the country participated. Their collections ranged from 56,000 to 4 million volumes. A total of 531 transactions with matching patron and librarian forms were analyzed in terms of 35 variables. The reliability of the instrumentwas found to be .81 overall-.89 for the patron form and. 76 for the librarian form. It was also found that patron report of success on factual questions did not differ significantly from results obtained from unobtrusive observation.

Quick and Easy Reference Evaluation

“…The Reference Transaction Assessment Instrument (RTAI) that grew out of WOREP was developed by Charles Bunge, Marjorie Murfin, and Gary M. Gugelchuk. 7 WOREP and the RTAI are currently available at http://worep.library.kent.edu. The great benefit of the RTAI is that it is standardized, it is statistically valid and reliable, the results are machine readable (thus saving time and resources on data entry), and the results can be compared to other reference service units.…”

Patron Online Catalog Success

“…The RTAI has been judged reliable and valid, has been utilized by many libraries evaluating reference services, and contains a section that provides information on whether the catalog was used in the process of answering a reference query. 10 In an analysis of preliminary results from the RTAI, two factors have emerged that appeared to undermine the results of reference service seriously. One factor was that the librarian was too busy and the other concerned the librarian's practice (either from habit or necessity) of directing or suggesting only rather than helping with the search.…”

“…For a copy of the RTAI questions, readers should consult the article published in College & Research Libraries by Marjorie Murfin and Gary Gugelchuk. 10 A useful feature of the RTAI was comparative data provided for other academic libraries that have participated in the RTAI. In our analysis, we will refer to the results for academic libraries of similar size for comparison purposes.…”

A linguistic analysis of question taxonomies

“…Many reference evaluation tools require that the subjects of questions be recorded (King, 1982;Murfin & Gugelchuk, 1987), the intention being to identify subjects on which it is difficult for reference librarians to answer questions, or to which reference librarians frequently cannot give accurate answers (Crews, 1988;Rothstein, 1964). Some of the more recent work in developing standards for evaluating digital reference services suggests that the subject of a question is an important component of data to be collected about the types of questions received by a service (McClure, Lankes, Gross, & Choltco-Devlin, 2002;White, 2001).…”