Assessing research performance in UK universities using the case of the economics and econometrics unit of assessment in the 1992–2014 research evaluation exercises

Johnston, James; Reeves, Alan

doi:10.1093/reseval/rvw021

Cited by 8 publications

(12 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The first relates to institutional expectations. Research groups that perform below their universities' expectations may be candidates to be pulled out of the next E&E UOA (Johnston and Reeves 2017). One of the consequences of performing less well than expected may be a cut in funding or, very likely, no funding at all (Lee et al 2013;Sayer 2015).…”

Section: Resultsmentioning

confidence: 99%

An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK

Johnston

Reeves

2018

High Educ

Self Cite

View full text Add to dashboard Cite

This study charts the socio-geographic fragmentation of the economics discipline in the UK. It is revealed that the marriage of economics teaching and research is now limited to elite institutions located mainly in the south of the UK. None of the UK's new (post-1992) universities submitted to the Economics and Econometrics (E&E) unit of assessment (UOA) in 2014, the UK's most recent research evaluation exercise (REE). Lower REE scores are shown to be associated with higher withdrawal rates from the next E&E UOA and subsequent undergraduate economics programme closure. Universities that exit the E&E UOA moving to the Business and Management (B&M) UOA appear to benefit in the form of higher REE scores. Though restricted to the experience of one academic discipline in one country, the lessons from this study yield important insights into how the results of REEs can be a principal driver of change in higher education.

show abstract

Section: Resultsmentioning

confidence: 99%

An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK

Johnston

Reeves

2018

High Educ

Self Cite

View full text Add to dashboard Cite

show abstract

“…In contrast, one can also argue that university administrators may follow the logic of the REF exercise by allocating more resources to the most efficient users (i.e. Johnston and Reeves (2017) examined the economics and econometrics submissions of HEIs for the assessment periods in 1992, 1996, 2001, 2008 and 2014 and found that research groups which perform below universities' expectations are pulled out of the next submission periods. 14 This strategic move is also in line with the economic theory of investing more in a given sector in which there is higher marginal expected return compared to other sectors.…”

Section: Some Potential Implications For the Heismentioning

confidence: 99%

“…Wilkinson, 2019). Johnston and Reeves (2017) examined the economics and econometrics submissions of HEIs for the assessment periods in 1992, 1996, 2001, 2008 and 2014 and found that research groups which perform below universities' expectations are pulled out of the next submission periods. In the previous implementation of the research excellence assessments, institutes were allowed not to return any submissions in some fields where there was limited research activity and/or the quality of research was perceived to be low.…”

Section: Some Potential Implications For the Heismentioning

confidence: 99%

Evaluating the potential effect of the increased importance of the impact component in the Research Excellence Framework of the UK

Pinar

Ünlü

2019

British Educational Res J

View full text Add to dashboard Cite

The UK's Research Excellence Framework (REF) is a system that is intended to evaluate the quality of the research produced by higher education institutes (HEIs) in the UK in three areas: quality of research outputs; impact of this research beyond academia; and research environment. For the next REF, the funding bodies have reviewed the importance of the three assessment elements and decided to increase the weight of 'impact' to 25% (from 20% in REF2014) and decrease the weight of 'outputs' to 60% (from 65% in REF2014). This article first examines the relevance of some factors for the quality of impact submissions in REF2014 and finds that larger submissions and institutes with higher external research income received better impact scores in the REF. The article then examines the units of assessment (UoAs) and HEIs that benefitted from the inclusion of the impact agenda as part of REF2014 by examining the distribution of the quality-related research (QR) funding in the 2017-2018 period and finds that the QR funding gap among different UoAs tends to decrease but the gap among HEIs in most of the UoAs increased. With the increased importance of the impact agenda as a criterion for funding bodies, it is expected that research income will be concentrated in fewer universities in the future, with the increased importance of non-academic impact. This article also discusses some of the gaming strategies and long-term investment priorities that HEIs may engage in based on the new submission rules of the next REF.Notes: The QR funding (15-20-65) and QR funding (15-25-60) columns give total QR funding received by HEIs based on the 15%, 20% and 65% weights and the 15%, 25% and 60% weights given to the environment, impact and output, respectively. Total gains/losses are the differences between QR funding (15-20-65) and QR funding (15-25-60) columns.Effect of the impact component in the REF 153

show abstract

“…The development of a comprehensive and valid research evaluation measure is a crucial precondition for assessing the performance of individual faculty members in academic institutions for promotion and reward purposes ( Kirkpatrick and Locke, 1992 ). Research evaluation encompasses two main distinct groups of stakeholders: first, the government, as the primary research funder, and second, universities, which do the actual research ( Johnston and Reeves, 2017 ).…”

Section: Introductionmentioning

confidence: 99%

A Framework to Assess the Citation Performance of Complex Innovation Systems

Ronda-Pupo

2021

Front. Res. Metr. Anal.

View full text Add to dashboard Cite

Scientometric indicators are useful to evaluate the relevance of scientific research, to prepare rankings, and to evaluate and inform research policies. That is why the choice of appropriate indicators is a matter of primary concern. This article aims to introduce a framework to decide the appropriate type of indicator for assessing the citation-based performance of complex innovation systems. The framework is two-fold: First, it brings the methodology to decide when the use of standard average based indicators is granted, and when scale-invariant indicators are mandatory. Second, it provides the procedures to build scale-invariant indicators to assess the relative impact of complex innovation systems. The framework is validated empirically through the evaluation of the relative impact of the Chilean science system in 2017. The result suggests that the Chilean science system has characteristics of a complex innovation system such as the distribution of citations fits to a power law with an exponential cutoff −2.77±0.09 and a power-law correlation between the size of the system and its impact 1.29±0.11. Furthermore, the framework shows to be efficient to compare fields of vastly different sizes.

show abstract

Assessing research performance in UK universities using the case of the economics and econometrics unit of assessment in the 1992–2014 research evaluation exercises

Cited by 8 publications

References 16 publications

An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK

An investigation into the role played by research assessment in the socio-geographic fragmentation of undergraduate economics education in the UK

Evaluating the potential effect of the increased importance of the impact component in the Research Excellence Framework of the UK

A Framework to Assess the Citation Performance of Complex Innovation Systems

Contact Info

Product

Resources

About