All that glitters isn’t gold

Abstract

Purpose Digital collection assessment has focused mainly on evaluating systems, metadata and usability. While use evaluation is discussed in the literature, there are no standard criteria and methods for how to perform assessment on use effectively. This paper asserts that use statistics have complexities that prohibit meaningful interpretation and assessment. The authors aim to discover the problems inherent in the assessment of digital collection use statistics and propose solutions to address such issues. Design/methodology/approach This paper identifies and demonstrates five inherent problems with use statistics that need to be addressed when doing assessment for digital collections using the statistics of assessment tools on local digital repositories. The authors then propose solutions to resolve the problems that present themselves upon such analysis. Findings The authors identified five problems with digital collection use statistics. Problem one is the difficulty of distinguishing different kinds of internet traffic. Problem two is the lack of direct correlation of a digital item to its multiple URLs, so statistics from external web analytics tools are not ideal. Problem three is the analytics tools’ inherent bias in statistics that are counted only in the positive way. Problem four is the different interaction between digital collections with search engine indexing. Problem five is the evaluator’s bias toward simple growing statistics over time for surmising a positive use assessment. Because of these problems, statistics on digital collections do not properly measure a digital library’s value. Practical implications Findings highlight problems with current use measures and offer improvements. Originality/value This paper identifies five problems that need to be addressed before a meaningful assessment of digital collection use statistics can take place. The paper ends with a call for evaluators to try to solve or mitigate the stated problems for their digital collections in their own evaluations.

Books in a bubble.

“…Measuring the impact of a digital collection is challenging. (Perrin et al 2017) describe the complexities of understanding the data generated by Google Analytics of a DSpace environment. Other authors focus more on the stronger and weaker points of the library collection; an example is the work by (Hyödynmaa, Ahlholm-Kannisto, and Nurminen 2010) on collection mapping in Tampere University Library.…”

Toward a definition of digital object reuse

“…Assessing use of digital objects tends to be quantitative and relatively straightforward; examples of digital object use assessment metrics include views, viewing time, clicks, downloads, sessions and other web analytics; printing; altmetrics (e.g. downloads, saves in citation managers, recommendations and shares); video coding and annotation; and clones of code or data (Baughman et al , 2018; Carpenter et al , 2016; Faniel and Yakel, 2011; Fouseki and Vacharopoulou, 2013; Frank et al , 2016; GitHub and Various Contributors, 2018; Ladd, 2015; He and Han, 2017; Perrin et al , 2017; White et al , 2018; O’Gara et al , 2018; Kelly et al , 2018).…”

Bringing IRUS to the USA: International Collaborations to Standardize and Assess Repository Usage Statistics

“…21 Perrin et al focused on five specific repository usage problems, including "difficulty of distinguishing different kinds of internet traffic," a "lack of direct correlation of a digital item to its multiple URLs," "the analytics tools' inherent bias in statistics that are counted only in the positive way," "the different interaction between digital collections with search engine indexing," and "evaluator's bias toward simple growing statistics over time for surmising a positive use assessment." 22 To overcome these issues, Perrin et al advocate for institutions to evaluate usage data through the lens of the "sessions or user perspective." 23 As noted by multiple studies, standardizing practices is a current gap in digital repository usage assessment.…”

“…22 To overcome these issues, Perrin et al advocate for institutions to evaluate usage data through the lens of the "sessions or user perspective." 23 As noted by multiple studies, standardizing practices is a current gap in digital repository usage assessment. However, some groups are beginning to address this problem.…”