Are leading papers of better quality? Evidence from a natural experiment

Abstract

European countries in which universities rely on public funding increasingly follow the lead of the United Kingdom and run Research Assessment Exercises. Given the subjective nature of such evaluations, some scientists prefer verifiable measures such as citation counts. This, however, also is prone to problems since the number of cites is correlated, among others, with the order of appearance in an issue. In particular, leading papers are more cited. It is, thus, difficult to assess whether they are of better quality, or whether this happens because they appear first in an issue. We make use of a natural experiment that was carried out by a journal in which papers are randomly ordered in some issues, while this order is at the editors' discretion in other issues. Our estimates suggest that approximately two thirds of the additional cites are due to going first, and one third to higher quality. JEL classification: A10.

Citations In Economics: Measurement, Uses, and Impacts

“…But is this because their visibility leads readers to cite them more, or because editors' decisions to place them in the lead reflects editorial beliefs about their quality, which are then validated in the market for research? Using a journal that ordered papers alphabetically (by author's surname) in some issues, by editorial choice in others, Coupé, Ginsburgh, and Noury (2010) showed that about two-thirds of a lead article's extra citations were due to its position, one-third to its inherent higher quality than subsequently placed articles in the same issue.…”

Does the John Bates Clark Medal Boost Subsequent Productivity and Citation Success?

“…Nevertheless, using citations as a proxy for quality is not without problems (for recent discussions, see Coupé, Ginsburgh, andNoury, 2010, andPiatti, 2011). For example, fields with a larger research population attract more citations (Arrow et al, 2011;Cole and Cole, 1971), and citations can be driven by fashion (van Dalen and Klamer, 2005).…”

What Influences Business Academics’ Use of the Association of Business Schools (ABS) List? Evidence From a Survey of UK Academics

“…In evaluating research quality there has been an observable shift internationally towards utilizing more rankings‐based appraisal methods as a means to measuring the quality of research, with these approaches being embedded in other methods of appraisal such as research assessment exercises (Coupé, Ginsburgh and Noury, ; Hicks, ). It is certainly the case that there is a widely held perception that rankings have become an intrusive element in academic life.…”