All power to the reviewers: British sociology under two-level supervision of the Research Excellence Framework

Abstract

Our study investigates the impact of the British Research Assessment Exercise in 2008 and Research Excellence Framework in 2014 on the diversity and topic structure of UK sociology departments from the perspective of habitus-field theory. Empirically, we train a Latent Dirichlet allocation on 819,673 abstracts stemming from the journals in which British sociologists submitted at least one paper in the Research Assessment Exercise 2008 or Research Excellence Framework 2014. We then employ the trained model on the 4822 papers submitted in the Research Assessment Exercise 2008 and 2014. Finally, we apply multiple factor analysis to project the properties of the departments in the topic space. Our topic model uncovers generally low levels of research diversity. Topics with global reach related to political elites, demography, knowledge transfer, and climate change are on the rise, whereas locally constrained research topics on social problems and different dimensions of social inequality get less prevalent. Additionally, some of the declining topics are getting more aligned to elite institutions and high ratings. Furthermore, we see that the associations between different funding bodies, topics covered, and specialties among sociology departments changed from 2008 to 2014. Nonetheless, topics aligned to different societal elites are found to be associated with high Research Assessment Exercise/Research Excellence Framework scores, while social engineering topics, postcolonial- and cultural-related, as well as more abstract topics are related to lower Research Assessment Exercise/Research Excellence Framework scores.

Are evaluative bibliometrics neoliberal? A historical and theoretical problematization

“…Overall, it is a highly visible and influential system, which has been debated and studied intensively. For example, recent studies have argued that the REF has led to a homogenization of research in several fields (Pardo-Guerra, 2022), yet as shown by Wieczorek et al. (this volume), the concrete epistemological effects of the system are difficult to determine.…”