Shall we chat? A statistical case study of chat reference utilization

Abstract

Purpose This paper aims to evaluate the performance of a chat reference service implemented at an academic library in a private liberal arts college by gauging its impact on other forms of reference service in terms of usage volume, with a focus on research-related face-to-face reference questions. Design/methodology/approach Two statistical methods are used, namely, the difference-in-differences method and a simple moving average time series analysis, to analyze both the short-term and long-term impact brought by chat reference. Findings This study finds that the usage volume of the traditional face-to-face reference is significantly affected by chat reference in its first service year. The long-term analysis suggests that chat reference volume displays a significant declining trend (−2.06 per cent academic month) since its implementation. Yet, its usage volume relative to other reference services remains stable over time. Originality/value The findings in this case study will be of value to libraries with similar scale and institutional features that are also interested in assessing their chat reference service. In addition, this paper is the first to apply the difference-in-differences approach in the field of library science, and the two statistical methods adopted in this case study can be readily adapted and applied to other similar volume-based library assessment projects.