The rank boost by inconsistency in university rankings: Evidence from 14 rankings of Chinese universities

Abstract

University ranking has become an important indicator for prospective students, job recruiters, and government administrators. The fact that a university rarely has the same position in different rankings motivates us to ask: To what extent could a university’s best rank deviate from its “true” position? Here we focus on 14 rankings of Chinese universities. We find that a university’s rank in different rankings is not consistent. However, the relative positions for a particular set of universities are more similar. The increased similarity is not distributed uniformly among all rankings. Instead, the 14 rankings demonstrate four clusters where rankings are more similar inside the cluster than outside. We find that a university’s best rank strongly correlates with its consensus rank, which is, on average, 38% higher (towards the top). Therefore, the best rank usually advertised by a university adequately reflects the collective opinion of experts. We can trust it, but with a discount. With the best rank and proportionality relationship, a university’s consensus rank can be estimated with reasonable accuracy. Our work not only reveals previously unknown patterns in university rankings, but also introduces a set of tools that can be readily applied to future studies.

Understanding Chinese science: New scientometric perspectives

“…The contributions in this special issue primarily use quantitative analytic approaches, showing an impressive range of methods and techniques. Several articles adopt classical scientometric indicators such as coauthorship and citations, whereas some apply newly developed ranking indicators such as rank-biased overlap (Chen et al, 2021). Some combine bibliometrics and science visualization techniques such as science overlay maps, clustering, and heatmaps, whereas others develop innovative network analytics for identifying problems and solutions from scientific documents.…”

A Comprehensive Evaluation Framework to Assess the Sustainable Development of Schools within a University: Application to a Chinese University

“…Since the 1980s, the practice of ranking higher education institutions has become widespread among the popular press [7]. At present, different organizations have various strategies for assessing and comparing the performance of higher education institutions across a country or the world [8,9]. Some rankings mainly use a single index, such as the Nature Index (Ni), which only considers an institution's research productivity and impact via the number of publications in a select group of prominent natural-science journals [10].…”

Become a better you: correlation between the change of research direction and the change of scientific performance

“…The availability of large scale data set and computational tools makes it easier than ever to quantitatively probe the science at different scales, from papers (Wang et al, 2013;Hu et al, 2020;Chen et al, 2021a;Mukherjee et al, 2017) to individual scientists (Way et al, 2017;Liu et al, 2018;Bu et al, 2018;Zhang et al, 2017), and from research teams (Wu et al, 2019;Ma et al, 2020;AlShebli et al, 2018) to institutions and nations (King, 2004;Liu et al, 2020;Zhao et al, 2020;Zuo & Zhao, 2018;Chen et al, 2021c;Huang et al, 2020a). As the development of science is driven by scientists' involvements in different research topics, it is crucial to understand how they decide their research directions and what are the consequences of their collective decisions.…”