The Herrero-Villar approach to citation impact

Abstract

This paper focuses on the evaluation of research units in a scientific field in terms of size-independent indicators of citation impact when size is measured by the number of publications. There are well-known procedures in this context that evaluate each research unit based upon a partition of the field citations into a set of ordered categories, and an external weighting system that determines the relative importance of each category. These conventional procedures are seen to share a common additive structure. We introduce here a new ranking procedure -the HV procedure, after Herrero & Villar (2013)-and compare it within a common setting to a number of additive evaluation rules currently used in practice. Given a set of ordered categories, the HV procedure measures the performance of the different research units in terms of the relative probability of getting a greater citation impact. The HV method provides a complete, transitive and cardinal evaluation without recurring to any external weighting scheme. Using a large dataset of publications in 22 scientific fields assigned to 40 countries, we compare the performance of several additive evaluation rules -the Relative Citation Rate, four percentile-based ranking procedures, and two average-based high-impact indicators-and the corresponding HV procedures under the same set of ordered categories. Comparisons take into account re-rankings, and differences in the outcome variability measured by the coefficient of variation, the range, and the ratio between the maximum and minimum index values.Acknowledgements. This is the second version of a Working paper with the same title published in this series in December 2016. The dataset has been acquired thanks to a

Total number of papers and in a single percentile fully describes research impact—Revisiting concepts and applications

“…In terms of citations, the two groups are defined depending on whether they are above or below a certain citation threshold-the issue of citation ties has been discussed previously (Schreiber, 2013;Waltman & Schreiber, 2013). This dichotomous classification of papers (Albarrán, Herrero et al, 2017;Bornmann, 2013) leads to the important notion that "dichotomous procedures rely on the idea that only the upper part of the distribution matters" (Albarrán et al, 2017, p. 628). Consequently, in formal terms, dichotomous indicators do not consider papers that are excluded by the criterion.…”

Dealing with categorical data in a multidimensional context: The multidimensional balanced worth

“…Freeman (1977), Wasserman & Faust (1994), or Newmann (2003) apply it to the study of centrality in networks. Pinsky & Narin (1976), Liebowitz & Palmer (1984), Palacios-Huerta & Volij (2004) use it to evaluate the relevance of the journals in citation analysis, and Albarrán et al (2017) to compare the performance of countries in different disciplines based upon the number of citations. Keener (1993), or Slutzski & Volij (2005 apply it to the ranking of teams in competitions.…”