Defining standards for rankings: An investigation of global university rankings according to the Berlin Principles

Abstract

The role of universities as the engines of knowledge‐based economies drives global internationalisation of higher education. This contributes to a competitive environment where higher education rankings indicate market value. Even though rankings are influential and are used a lot, ranking systems have been heavily critiqued. One of the problems is that there are few if any external checks on how rankings are created. The purpose of the study on which this article reports was to evaluate ranking systems. Within the scope of the study, we have sought to reveal to what extent current ranking systems comply with the Berlin Principles—prepared to create certain rules for rankings, and to ensure that rankings represent quality. A document analysis of publicly available documents online was carried out together with a review of printed and electronic publications on ranking systems. An evaluation form was prepared and used in this study for field experts to fill in. Findings show that ranking systems comply with the Berlin Principles in terms of methodology, transparency, and acceptability at a level that ranges from moderate to low. Overall, rankings do not consider differences in higher education and are not transparent about the processes by which rankings are developed. Rankings should for this reason be interpreted carefully and methodological weaknesses of rankings that can lead to false inferences should be recognised.