How Should Research Performance Be Measured? A Study of Swedish Economists*

Abstract

Billions are allocated annually to university research. Increased specialisation and international integration of research and researchers have sharply raised the need for comparisons of performance across fields, institutions and individual researchers. However, there is still no consensus regarding how such rankings should be conducted and what output measures to use. We rank all full professors in a particular discipline, economics, in one country using seven established, and some of them commonly used, measures of research performance. We show both that the rank order varies greatly across measures, and that depending on the measure used the distribution of total research output is valued very differently.

The Evolution of the Scientific Productivity of Highly Productive Economists

“…Except for the fact that total mobility is generally less equalizing in all cohorts, with the consequence that aggregate productivity inequality for the sample as a whole is now 4%—rather than 3%—greater than productivity inequality in the first period, the results are strikingly similar to the ones discussed in the paper. This contrasts with the main finding of Henrekson and Waldenström (2011), suggesting large discrepancies between seven measures of productivity in terms of both the rank order of Economics professors in Sweden and the absolute differences between their performances.…”

“…Coupé, Smeets, and Warzynski (2006) use the average of the rankings based on 12 different weighting schemes computed by Coupé (2003). In order to assess the different degree of elitism involved, Henrekson and Waldenström (2011) display the cumulative distribution of the weights attributed in three important measures of journal quality. The university listing, together with information on the number of faculty members, the total of publications in each class, and the two productivity indices we use can be found in Appendix B of Carrasco and Ruiz‐Castillo (2012).…”

An Evaluation of the Greek Universities’ Economics Departments

“…The most well‐known are those of Ben‐David () for Israel, Cokgezen () for Turkey, Ruan and Tol () and Tol () for Ireland, Clerides et al . () for the UK, and Henkerson and Waldenstrom () for Sweden. For a comparison of rankings and the institutional framework between Italy and Canada, see also Pelloni ().…”

“…In addition, especially for the Economics Departments, such classification/ranking studies are still very limited. The most well-known are those of Ben-David (2010) for Israel, Cokgezen (2006) for Turkey, Ruan and Tol (2008) and Tol (2008) for Ireland, Clerides et al (2011) for the UK, and Henkerson and Waldenstrom (2011) for Sweden. For a comparison of rankings and the institutional framework between Italy and Canada, see also Pelloni (2009).…”

Economists, Research Performance and National Inbreeding: North Versus South

“…Oswald () argues on the issue: ‘ if the criterion is intellectual impact measured by citations, in this sample it was better to publish the top article in an issue of the Oxford Bulletin of Economics and Statistics than to publish all of the bottom‐four papers in an issue of the American Economic Review ’ . On the other hand, more recently published papers, as those on Ireland (Ruan and Tol, ; Tol, ), on Israel (Ben‐David, ), on Sweden (Henkerson and Waldenström, ) and on Greece (Katranidis et al . ) rely on bibliometric databases (Web of Science, Google Scholar and Scopus) and consider directly the scientific impact of each paper separately, i.e.…”