Challenging Valuations: How Rankings Navigate Contestation

Abstract

Accounts of why rankings are pervasive features of the modern world focus mostly on their properties as valuation devices that, upon entering the public sphere, exert pressure on the ranked. In doing so, however, research tends to overlook the important role played by the different types of organizations that produce rankings. To remedy this, the article draws from a qualitative study consisting of semi-structured interviews with members of these organizations to show that they put a great deal of effort into addressing and responding to different kinds of criticism. Working towards building and maintaining the credibility of rankings is thus revealed to require constant attention by their producers, who devise multiple procedures and rhetorical strategies to this end.

The discursive resilience of university rankings

“…Although continuously targeted by challenges that emphasize negative effects or methodological shortcomings, university rankings seem to be quite resilient towards this criticism. Extending previous accounts (Ringel et al, 2021; see also Ringel, 2021), our tentative analysis suggests that rankers mobilize two modes of responding to criticism. We refer to the first mode as (3) defensive responses and to the second mode as (4) confident responses to critique of rankings.…”

“…However, building on recent studies (cf. Barron, 2017;Kaidesoja, 2022;Lim, 2018) and our previous research (Ringel et al, 2021; see also Brankovic et al, 2022;Ringel, 2021), we contend that in order to fully understand the proliferation of university rankings, we also need to account for meso-level processes. Only then are we able to see how university rankings can prevail despite being exposed to frequent criticism.…”

“…(3a) Although rankers have become proficient in leveraging public relations expertise to transform league tables into spectacular public performances (Brankovic et al, 2018;Ringel, 2021), they are also accomplished in trivializing university rankings by downplaying their influence. If deemed necessary, rankers point to their competitors or other types of university evaluation.…”

“…Second, we emphasized rankers' proficiency when dealing with criticism. In doing so, we neglected potential tensions producers of rankings are facing, especially between the affordances of scholarly sophistication and rigor, on the one hand, and the necessity of providing simplified explanations and aesthetically appealing visualizations for lay audiences, on the other (Ringel, 2021). Third, it is an open question whether the division of discursive labor between rankers and critics that we found in higher education also contributes to the success of rankings in other fields-for instance, in healthcare, international development, or the arts.…”

Spreading the gospel: Legitimating university rankings as boundary work

“…Based on these insights, we suspect that a similar kind of boundary work could be going on in other domains, such as in the case of nation-states, corporate, or artist rankings. In all of these cases, rankings face criticism, which gives their producers cause to be worried about the legitimacy of their products and drives them to directly engage with their various critics (Ringel 2021). We therefore believe that our analysis provides a useful blueprint for understanding boundary work as an underrated condition for the recent success of rankings, and possibly other contested devices of evaluation, not just in the university sector but also more generally.…”