The Differing Meanings of Indicators Under Different Policy Contexts. The Case of Internationalisation

Abstract

In this chapter we build upon Moed's conceptual contributions on the importance of the policy context when using and interpreting scientometric indicators. We focus on the use of indicators in research evaluation regarding internationalisation policies. The globalization of higher education presents important challenges to institutions worldwide, which are confronted with tensions derived from the need to respond both, to their local necessities and demands while participating in global networks. In this context, indicators have served as measures for monitoring and enforcing internationalisation policies, in many cases interpreting them regardless of the policy context in which they are enforced. We will analyse three examples of indicators related to internationalisation. The first one is about international collaborations, under the assumption that a greater number of internationally co-authored publications will benefit a national science system as it will result in higher citation impact. The second one relates to the promotion of English language as the dominant language of science. The third case analyses how different policy contexts shape the selection and construction of indicators, sometimes in a partial way which does not properly reflect the phenomenon under study. The examples illustrate that the interpretation and policy implications of the 'same' S&T indicators differ depending on specific contexts.

Connecting Scientometrics: Dimensions as a Route to Broadening Context for Analyses

“…This comment is, however, a naïve one, since it is likely not merely a choice not to work together (even on items of common interest such as SDGs), but a more fundamental systemic issue. We suggest that may be a variety of drivers behind this lack of engagement including: geographic separation; developing rather than developed research infrastructure to support international collaboration such as dedicated travel bursaries, international fellowship schemes, and technological infrastructures to support online collaboration; lack of links at a geopolitical level that either hinder or simply do not promote the establishment of links and relationships through researcher mobility that can support ongoing research relationships (Chikanda et al, 2016 ; Robinson-Garcia et al, 2019 ; Wagner, 2019 ; Robinson-Garcia and Ràfols, 2020 ; OECD, 2021 ). However, from our analysis, Figure 7 makes it clear that, even when viewed from the perspective of low-income countries themselves, engagement with other low-income countries is relatively lower—and on topics that are important and potentially highly meaningful to low-income countries.…”

Multilingualism of social sciences

“…One challenge in assessment is that international excellence or publishing is too often equated with English language publishing (Neylon, 2019;Robinson-Garcia & Rafols, 2020), or even as publishing in journals indexed in the Web of Science or Scopus. Using publication language (especially English) as a criterion may compromise fair assessment of researchers and units with strong orientation and mission toward locally relevant research or societal interaction.…”

Valuation regimes in academia: Researchers’ attitudes towards their diversity of activities and academic performance

“…This is due to the well-known biases mainstream bibliometric databases have in regard to language (van Leeuwen et al 2001), disciplinary coverage (Archambault et al 2006) and peripheral or regional knowledge (Rafols et al 2015). A non-discriminatory use of metrics in different contexts undermines the performance of scientists in fields and regions which do not adjust to the expectations in which bibliometric indicators are built (Gläser and Laudel 2007;Ràfols et al 2016;Robinson-Garcia and Ràfols 2020).…”