Committees or Markets? An Exploratory Analysis of Best Paper Awards in Economics

Abstract

Despite the general usefulness of citations as a sort of test of the value of one’s work in the marketplace of ideas, journals and publishers tend to use alternative bases of judgment, namely committees, in selecting candidates for the conferral of journals’ best paper awards. Given that recognition—sometimes in the form of compensation and on other occasions in the form of awards—in academe is geared toward incentivizing the production of impactful research and not some less desirable goal or outcome, it is important to understand the sensitivity in the outcomes of best paper award selection processes to the types of processes used. To that end, this study compares the selection of best paper awards for journals affiliated with several of the world’s top economic associations by committees to a counterfactual process that is based on citations to published studies. Our statistical exploration indicates that in most cases and for most awards, the most cited paper was not chosen. This requires further discussion as to the core characteristics that quantitatively represent the highest impact.

“Best Paper” awards lack transparency, inclusivity, and support for Open Science

“…Studies in Computer Science [ 39 ] and Ecology and Evolution [ 11 ] have revealed opaque award criteria and processes. In Economics, a study has found a mismatch between subjective jury decisions and article citation impacts, similar numbers of men and women among the winners, and affiliations dominated by the United States institutions [ 40 ]. In Engineering, most winning authors were not native English speakers [ 41 ].…”

Best Paper awards lack transparency, inclusivity, and support for Open Science

“…For instance, this is the case in Computer Science 33 . In Economics, studies have found a notable mismatch between subjective jury decisions and article citation impacts, similar numbers of men and women among the winners, and affiliations dominated by the US institutions 34 . In Engineering journal awards, most winning authors were not native English speakers 35 .…”