Can the impact of grey literature be assessed? An investigation of UK government publications cited by articles and books

Abstract

Grey literature encompasses a range of relatively informal textual outputs that are not indexed in citation databases. Although they are usually ignored in research evaluations, it is important to develop methods to assess their impact so that their contributions can be recognised, and successful types of grey literature can be encouraged. This article investigates the extent to which 97,150 UK government publications were cited by Scopus articles and Google Books during 2013-2017 in eleven broad subject areas. A method was used to semiautomatically extract citations to the UK government publications from articles and books with high recall and precision. The results showed that Scopus citations are more common than Google Books citations to UK government publications, especially for older documents, and for those in Healthcare, Education and Science. Since the difference is not huge, both may provide useful grey literature impact data.

Assessing the Capability of Government Information Intervention and Socioeconomic Factors of Information Sharing during the COVID-19 Pandemic: A Cross-Country Study Using Big Data Analytics

“…For example, the web pages that cited a health authority’s website URL were counted. Web-traffic volume data are commonly used to evaluate the impact of websites [ 24 ]. Compared to normalized or scaled data, such as search engine query volumes, raw web traffic data are more useful for tracking and predicting public behavior, as they reflect public interest in real time [ 16 , 25 , 26 ].…”

An Institutional Repository Publishing Model for Imperial College London Grey Literature

“…16 Authors of grey literature may wish to understand or demonstrate impact of their works, but the lack of a centralised grey literature citation source inhibit this on a practical and widespread scale. Whilst others have proposed bespoke methods for calculating grey literature impact, 17,18 the computational nature of these methods are not feasible for the authors in our project accustomed to using services such as Google Analytics for metrics. In attempting to support authors with some form of metric; easy to access download, citations and alternative metrics have been adapted or built into the IR.…”

Google Books, Scopus, Microsoft Academic and Mendeley for impact assessment of doctoral dissertations: A multidisciplinary analysis of the UK

“…Google Books citations are up to twice as numerous as WoS citations for journal articles (Kousha & Thelwall, 2009) and up to four times as numerous as Scopus citations to humanities books (Kousha, Thelwall, & Rezaie, 2011). Although it seems that no study has investigated citations from Google Books to dissertations, it is a useful source of impact assessment for gray literature and queries can be automated (Bickley, Kousha, & Thelwall, 2019). Hence, Google Books might also provide an automatic source of citations to doctoral dissertations, and this may be especially valuable in book-based fields where edited books and monographs are important.…”