Incentives for Replication in Economics

Abstract

and participants in the 2016 BITSS annual meeting. Ada Kwan and Alexandra Wall provided excellent research assistance. The authors have no material or financial interests in the results of the paper. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research. NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.

Introducing the Replication Studies section

“…This might be because replication studies are often treated as insufficiently novel for publication. In a survey, Galiani et al (2017) found that out of the 204 editors and co-editors of 11 top journals in economics, only 29% would publish replication studies confirming existing results.…”

“…The literature suggests two reasons for the lack of replication studies (Dewald et al, 1986;Galiani et al, 2017;Vilhuber, 2020):…”

Revisiting the impact of per‐unit duties on agricultural export prices

“…It is also important to validate the findings and detect potential biases in the existing literature. However, these exercises are not exempt from threats in that the replication itself is subject to incentives that may lead to biases, such as the “overturn bias”, where authors report false positives or claim mistakes in the original analysis without solid justifications (Galiani, Gertler, & Romero, 2017).…”

A Review of Multisite Replication Projects in Social Psychology: Is It Viable to Sustain Any Confidence in Social Psychology’s Knowledge Base?

“…No strong conclusions can be drawn regarding possible editorial bias, but we think it worth mentioning, in part because publication bias is widely assumed to contribute to the inflated effect sizes among original findings. Galiani et al (2017) surveyed journal editors in economics and found that they preferred to publish failed rather than successful replications. Such a preference would be understandable, insofar as journal editors seek to preserve their much-sought-after journal space for new information that advances the field.…”