What do citation counts measure? An updated review of studies on citations in scientific documents published between 2006 and 2018

Abstract

The purpose of this paper is to update the review of Bornmann and Daniel (2008) presenting a narrative review of studies on citations in scientific documents. The current review covers 41 studies published between 2006 and 2018. Bornmann and Daniel (2008) focused on earlier years. The current review describes the (new) studies on citation content and context analyses as well as the studies that explore the citation motivation of scholars through surveys or interviews. One focus in this paper is on the technical developments in the last decade, such as the richer meta-data available and machine-readable formats of scientific papers. These developments have resulted in citation context analyses of large datasets in comprehensive studies (which was not possible previously). Many studies in recent years have used computational and machine learning techniques to determine citation functions and polarities, some of which have attempted to overcome the methodological weaknesses of previous studies. The automated recognition of citation functions seems to have the potential to greatly enhance citation indices and information retrieval capabilities. Our review of the empirical studies demonstrates that a paper may be cited for very different scientific and non-scientific reasons. This result accords with the finding by Bornmann and Daniel (2008). The current review also shows that to better understand the relationship between citing and cited documents, a variety of features should be analyzed, primarily the citation context, the semantics and linguistic patterns in citations, citation locations within the citing document, and citation polarity (negative, neutral, positive).

On the Possibilities of Evaluating Properties of Scientific Documents on the Basis of their Citations Count (or again: What Property is Reflected by Citations Count par excellence, after all?). Part 1: Value

“…The verb "to measure" that is widely used as applied to citations count Tahamtan & Bornmann, 2019) might be a source of confusions in itself. The fact is that, generally, "measure" relates to direct measurement, while in case of using bibliometric indicators we deal with indirect, mediated evaluation (Motylev, 1983).…”

“…Particular attention has been paid to the article by I. Tahamtan & L. Bornmann (2019), as this is the most recent publication on this topic, representing a solid 50-page review. The method is interpretation of above-mentioned recent publications; herewith the special attention is paid to the terminology.…”

“…The key question "What do citation counts measure?" which is, in fact, the title of two reviewing papers Tahamtan & Bornmann 2019) itself leads to plentiful confusions of concepts that prevent experts from understanding each other. To me, it is natural to expect that exactly the properties of the cited documents reflected by the citations counts are going to be the subject matter of a paper with such a title.…”

“…However, the recent reviewing papers Tahamtan & Bornmann, 2019) focused not on the properties of cited documents, but on "motivations, functions, and causes of references in scientific communication" (Tahamtan & Bornmann, 2019). These issues relate to different objects other than documents, viz., correspondingly, to the authors (who are motivated), references (which have their functions) and the citer's comprehension of the document being cited (when a citer takes into account both his own goals and a content of a document or its fragment he is caused either to cite or not to cite).…”

How much is too much? The difference between research influence and self-citation excess

“…Reasons for self-citation include: "the cumulative nature of individual research, the need for personal gratification, or the value of self-citation as a rhetorical and tactical tool in the struggle for visibility and scientific authority" (Fowler and Aksnes 2007). Such reasons imply behaviors that conform both to Mertonian norms and also to the social constructivist interpretation of citation theory (Merton 1942;Kaplan 1965;Moravcsik and Murugesan 1975;Gilbert 1977;Cozzens 1989;Bonzi and Snyder 1991;White 2001;Bornmann and Daniel 2008;Davis 2009;Erikson and Erlandson 2014;Bornmann 2018, 2019;Aksnes et al 2019). Small (2004) offered a synthesis of motives for citation, in which citations serve a dual function as "vehicles of peer recognition and constructed symbols for specific original achievements."…”