Unique, but still best practice? The Research Excellence Framework (REF) from an international perspective

Abstract

Inspired by The Metric Tide report (2015) on the role of metrics in research assessment and management, and Lord Nicholas Stern's report Building on Success and Learning from Experience (2016), which deals with criticisms of REF2014 and gives advice for a redesign of REF2021, this article discusses the possible implications for other countries. It also contributes to the discussion of the future of the REF by taking an international perspective. The article offers a framework for understanding differences in the motivations and designs of performance-based research funding systems (PRFS) across countries. It also shows that a basis for mutual learning among countries is more needed than a formulation of best practice, thereby both contributing to and correcting the international outlook in The Metric Tide report and its supplementary Literature Review.

An introduction to achieving policy impact for early career researchers

“…Rather, our ability to successfully navigate contemporary issues and steer society towards a positive future is dependent on the ways in which scientific knowledge is applied in policy and practice (Cairney, 2016;Caplan, 1979;Evans et al, 2017;Shonkoff, 2000;Sutherland et al, 2004;Weiss, 1979). As a result scientists have come under greater societal and political pressure to more actively engage with decision-makers to facilitate knowledge exchange and the uptake of scientific knowledge into decision-making processes (Boswell and Smith, 2017;Lubchenco, 1998;Sivertsen, 2017;Stein, 2018). At the same time scientists are increasingly required to demonstrate the tangible and real-world impacts 1 arising from their research (Boswell and Smith, 2017;Sivertsen, 2017;Smith and Stewart, 2017;Stein, 2018).…”

“…As a result scientists have come under greater societal and political pressure to more actively engage with decision-makers to facilitate knowledge exchange and the uptake of scientific knowledge into decision-making processes (Boswell and Smith, 2017;Lubchenco, 1998;Sivertsen, 2017;Stein, 2018). At the same time scientists are increasingly required to demonstrate the tangible and real-world impacts 1 arising from their research (Boswell and Smith, 2017;Sivertsen, 2017;Smith and Stewart, 2017;Stein, 2018). …”

“…Indeed, some scholars may feel pressured to 'game' institutionalised impact frameworks (Murphy, 2017;Sivertsen, 2017;Smith and Stewart, 2017), or that current definitions and metrics of "impact" may not necessarily reflect the full scope of benefits derived from research (Stein, 2018). An in-depth exploration of these issues is outside of the scope of this article, but can be found elsewhere (Boswell and Smith, 2017;Marcella et al, 2017;Sivertsen, 2017;Smith and Stewart, 2017;Stein, 2018) Our intention for this manuscript is to offer practical guidance specifically for ECRs who are interested in engaging with policy and practice to achieve "impact". Existing literature has provided advice for ECRs on communication (Christian, 2018;Kuehne et al, 2014), stakeholder engagement (Duchelle et al, 2009), and diverse opportunities to engage with policy both inside and outside of academia (Bednarek et al, 2018;Petes and Meyer, 2018;Pietri et al, 2013).…”

Measuring Time-Dynamics and Time-Stability of Journal Rankings in Mathematics and Physics by Means of Fractional p-Variations

“…However, it must be said that some researchers consider that these criticisms on the metric-based systems must be analyzed in national contexts, since the usage of other procedures-such as peer-review-based systems-also has limitations. Indeed, it seems that the adequacy of any system needs to be examined in its national contexts to understand their motivations and design, as suggested in [23].…”

“…In general, as was clearly shown in [23], each country has developed its own method for research assessment, and a considerable effort has been put forth towards finding a convenient system in each national context. For example, the Spanish system for research evaluation is strongly based on impact metrics.…”