We use the UK 's 2014 Research Excellence Framework (REF) to study the attributes of top-scoring (four-star) publications in Economics and Econometrics. Although official documents contain aggregate scores for each institution, we show how these aggregates can be used to infer the score awarded by REF panellists to each publication. We demonstrate that this score responds to journal prestige as measured by the Thomson Reuters Article Influence Score. Several econometric analyses confirm the limited contribution of other publication attributes, such as the citation counts, to the awarded score, and publications in the top generalist and top five economics journals are awarded four stars unambiguously. We conclude that in large-scale evaluations such as the REF, peer reviews and bibliometrics should be viewed as complementary modes of assessment; the time-consuming task of peer reviews would be more cost-effective if targeted at publications whose quality cannot be classified unambiguously using bibliometrics.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.