Number of references: a large-scale study of interval ratios

Abstract

The paper presents a large-scale study (covering 26,998,764 items) of the development in the number of references over time in three document types (articles; reviews; notes) from seven fields (Arts and Humanities; Social Sciences; Computer Science; Mathematics; Engineering; Medicine; Physics and Astronomy). Using interval ratios instead of average numbers, the paper makes it possible to follow the development, and to locate the main causes of growth in the number of references over time. The results show significant differences between fields and document types. The number of references in journal articles and reviews are growing in all fields (except for the reviews in Arts and Humanities that remain stable over time), but at different pace; The number of references in notes are growing in some fields (again at different pace) and are stable in others. The observed growth is primarily caused by a drop in short reference lists and a corresponding increase in a bit longer and medium size reference lists. Long and very long reference lists remain much more stable in shares over time, and does therefore not contribute much to the observed growth. The results underline the importance of normalizing citation data, and for taking citation inflation into account when conducting citation analyses expanding different fields, document types, and longer time-periods.

Literary runaway: Increasingly more references cited per academic research article from 1980 to 2019

“…Yet, the inflection point was not explicitly specified. For now, we are sure that, for most disciplines, the inflection point did not occur in 2005 [9], nor in 2013 [1], nor in 2015 [10], and not even by 2019 (shown in Fig 2, also in [11]). It appears that the reference lists of journal articles have undergone a runaway process of ever-expanding increase.…”

“…In addition, researchers have speculated about other factors, including the advancement of teamwork and multiple authorships [5], pressure from reviewers [2], and the preferences of journal editors and readers [19]. Another limitation of this study is that our data come from a single source, namely, Journal Citation Reports, as opposed to others that consider Scopus, which is thought to have a wider coverage [10,11].…”

“…Studies have shown a trend for increasing numbers of references across different time windows in the fields of psychology [5], biochemistry [6,7], life sciences [2], polymer materials [8], and engineering [1]. Although a few studies have conducted multidisciplinary examinations [9][10][11], it is still not clear how citing behavior changes over longer periods of time, or what the current citing status is in major disciplines. Some researchers have proposed that the increase in the number of references is due to the increase in the total number of publications [3,6,10] and easy access to the literature since the year 2000 as the Internet and online academic publishing platforms become increasingly accessible [1,5,11], referred to hereafter as "millennium acceleration."…”

“…Although a few studies have conducted multidisciplinary examinations [9][10][11], it is still not clear how citing behavior changes over longer periods of time, or what the current citing status is in major disciplines. Some researchers have proposed that the increase in the number of references is due to the increase in the total number of publications [3,6,10] and easy access to the literature since the year 2000 as the Internet and online academic publishing platforms become increasingly accessible [1,5,11], referred to hereafter as "millennium acceleration." However, we are not aware of any study that has compiled a large dataset and conducted a thorough analysis to empirically test these hypotheses.…”

About right: references in Open Access EGU journals

“…The temporal increase in the number of references per papers varies among different disciplines (Sánchez- Gil et al, 2018). Furthermore, Nicolaisen and Frandsen (2021) showed that "there is a drop in short reference lists and a corresponding increase in a bit longer and medium size reference lists. Long and very long reference lists remain much more stable in shares over time, and does therefore not contribute much to the observed growth."…”