The Measurement of the Effect on Citation Inequality of Differences in Citation Practices across Scientific Fields

Abstract

This paper has two aims: (i) to introduce a novel method for measuring which part of overall citation inequality can be attributed to differences in citation practices across scientific fields, and (ii) to implement an empirical strategy for making meaningful comparisons between the number of citations received by articles in 22 broad fields. The number of citations received by any article is seen as a function of the article’s scientific influence, and the field to which it belongs. A key assumption is that articles in the same quantile of any field citation distribution have the same degree of citation impact in their respective field. Using a dataset of 4.4 million articles published in 1998–2003 with a five-year citation window, we estimate that differences in citation practices between the 22 fields account for 14% of overall citation inequality. Our empirical strategy is based on the strong similarities found in the behavior of citation distributions. We obtain three main results. Firstly, we estimate a set of average-based indicators, called exchange rates, to express the citations received by any article in a large interval in terms of the citations received in a reference situation. Secondly, using our exchange rates as normalization factors of the raw citation data reduces the effect of differences in citation practices to, approximately, 2% of overall citation inequality in the normalized citation distributions. Thirdly, we provide an empirical explanation of why the usual normalization procedure based on the fields’ mean citation rates is found to be equally successful.

The skewness of scientific productivity

“…10 In this case, on average 68.6% of all articles receive citations below the mean and account for 29.1% of all citations, while articles with a remarkable or outstanding number of citations represent 10% of the total, and account for approximately 45% of all citations. These 10 These results are of the same order of magnitude as those obtained for field citation distributions in the multiplicative case for different time periods, at different aggregation levels, and with a fixed or a variable citation window Li et al, 2013). For the fractional case, see Herranz and Ruiz-Castillo (2012a). aggregate results for 219 scientific sub-fields, as well as the corresponding standard deviations and coefficients of variation, are extremely similar as those found in this paper for the definition of individual productivity as mean citation per article per person and authors are classified in 30 broad fields (row III in Table 3).…”

“…Thirdly, the between-field mean productivity differences in our dataset can be attributed to idiosyncratic differences in production and citation practices. However, just as the similarities between field citation distributions at different aggregation levels have recently paved the way for meaningful comparisons of citations for articles belonging to heterogeneous fields (Crespo et al, 2013, and Li et al, 2013, the similarities documented in this paper between field productivity distributions open the possibility of establishing meaningful comparisons of productivity for authors belonging to heterogeneous fields. In order to explore this possibility in our case, we have normalized field productivity distributions by computing the ratio between mean productivities in every field and mean productivity in Chemistry & Chemical Engineering, taken as the reference field.…”

The comparison of classification-system-based normalization procedures with source normalization alternatives in Waltman and Van Eck (2013)

“…In particular, WVE use the measurement framework introduced in Crespo et al (2013a) where, given a classification system, the effect on citation inequality of differences in citation practices is captured by an IDCP between-group inequality term in a certain partition of the overall citation distribution by field and quantile, where IDCP stands for citation Inequality attributable to Differences in Citation Practices. The evaluation of any set of normalization procedures in terms of a given classification system can take a graphical or a numerical form.…”

Quantitative evaluation of alternative field normalization procedures

“…Given a classification system into a number of scientific fields, Crespo et al (2013a) introduces a simple model in which the number of citations received by an article is a function of two variables: the article's underlying scientific influence, and the field to which it belongs to. Consequently, the citation inequality in the all-fields case is the result of two factors: differences in scientific influence within homogeneous fields, and differences in citation practices across fields.…”

“…For high Π, the first term is expected to be small, while the second -capturing the skewness of science in the all-sciences case-is expected to be large. For details, see Crespo et al (2013a). 1. Normalization by sub-field average.…”