Transparency, Reproducibility, and the Credibility of Economics Research

Christensen, Garret; Miguel, Edward

doi:10.1257/jel.20171350

Cited by 299 publications

(73 citation statements)

References 339 publications

(325 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…It plays a critical role in controlling the impact of sampling error, questionable research practices, publication bias and fraud [1,2]. An increase in the effort to replicate studies has been argued to help establishing credibility within a field [3]. Moreover, replications allow us to test if results generalize to a different or larger population and help to verify the underlying theory [2,4,5] and its scope [6].…”

Section: Introductionmentioning

confidence: 99%

Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme

et al. 2020

View full text Add to dashboard Cite

The Defense Advanced Research Projects Agency (DARPA) programme ‘Systematizing Confidence in Open Research and Evidence' (SCORE) aims to generate confidence scores for a large number of research claims from empirical studies in the social and behavioural sciences. The confidence scores will provide a quantitative assessment of how likely a claim will hold up in an independent replication. To create the scores, we follow earlier approaches and use prediction markets and surveys to forecast replication outcomes. Based on an initial set of forecasts for the overall replication rate in SCORE and its dependence on the academic discipline and the time of publication, we show that participants expect replication rates to increase over time. Moreover, they expect replication rates to differ between fields, with the highest replication rate in economics (average survey response 58%), and the lowest in psychology and in education (average survey response of 42% for both fields). These results reveal insights into the academic community's views of the replication crisis, including for research fields for which no large-scale replication studies have been undertaken yet.

show abstract

Section: Introductionmentioning

confidence: 99%

Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme

et al. 2020

View full text Add to dashboard Cite

show abstract

“…At this stage, it is important to distinguish the quality of the management of the research process from the quality of the research itself. A stream of the economic literature focuses on the related problem of transparency and selection bias in methods and results in academic journals (Christensen and Miguel, 2018). These papers focus on the replicability of certain econometric methods, leading not only to practices such as cherry‐picking and publication biases but also to the failure of replication due to opacity in the research process (Ioannidis, 2005).…”

Section: Reproducible Researchmentioning

confidence: 99%

“…However, journal's policies may not ensure reproducibility of the research because either the shared numerical materials are provided with a low quality (McCullough, 2018) or because of human or resources constraints. Christensen and Miguel (2018) note that a large share of published papers has a data exemption, mostly for confidential reasons. Even if this might be an obstacle to the enforcement of those policies, it should not refrain good practices.…”

Section: Introductionmentioning

confidence: 99%

How to Make a Pie: Reproducible Research for Empirical Economics and Econometrics

Orozco

Bontemps

Maigné

et al. 2020

Journal of Economic Surveys

View full text Add to dashboard Cite

Empirical economics and econometrics (EEE) research now relies primarily on the application of code to datasets. Handling the workflow linking datasets, programs, results and finally manuscript(s) is essential if one wish to reproduce results, which is now increasingly required by journals and institutions. We underline here the importance of "reproducible research" in EEE and suggest three simple principles to follow. We illustrate these principles with good habits and tools, with particular focus on their implementation in most popular software and languages in applied economics.

show abstract

“…13 Overall, research endeavors in the context of Third-Republic France are bound to be hindered by the lack of a shared frame of reference that would not only ease researchers efforts by generating economies of scale, but also improve reproducibility and interoperability across scientific studies. 14,15 To alleviate these issues and empower research programs in this context, this article proposes a comprehensive geographic information system of Third-Republic France: the TRF-GIS. It provides annual nomenclatures (codifications and toponymies) along with shapefiles of administrative constituencies of metropolitan France (mainland France and Corsica) from 1870 to 1940-901 nomenclatures and 830 shapefiles in total.…”

Section: Background and Summarymentioning

confidence: 99%

Mapping the Third Republic. A Geographic Information System of France (1870–1940).

Gay

2020

SSRN Journal

View full text Add to dashboard Cite

This article describes a comprehensive geographic information system of Third-Republic France: the TRF-GIS. It provides annual nomenclatures and shapefiles of administrative constituencies of metropolitan France from 1870 to 1940, encompassing general administrative constituencies (départements, arrondissements, cantons) as well as the most significant special administrative constituencies: military, judicial and penitentiary, electoral, academic, labor inspection, and ecclesiastical constituencies. It further proposes annual nomenclatures at the contemporaneous commune level that map each municipality into its corresponding administrative framework along with its population count. The 901 nomenclatures, 830 shapefiles, and complete reproduction material along with primary sources of the TRF-GIS database are available at https://dataverse.harvard.edu/dataverse/TRF-GIS. regions and subdivisions), judicial constituencies (courts of appeal and of first instance), penitentiary constituencies, electoral constituencies (circonscriptions), academic constituencies (academies), labor inspection constituencies, and ecclesiastical constituencies (dioceses). The TRF-GIS further proposes annual nomenclatures at the contemporaneous commune level that map each municipality into its corresponding administrative framework along with its population count. The construction of TRF-GIS shapefiles uses a methodology different to the manual vectorization of georeferenced historical maps. 7, 13, 16, 17 It first reconstructs the administrative framework contemporaneous communes were embedded in, then aggregates these units into relevant constituencies using modern communes shapefiles. Despite some limitations, this method offers substantial advantages: not only it provides more precise results than existing outputs, but, more importantly, it also enables to circumvent the fundamental problems of non-existence of annual historical maps and of time and resource investments associated with manual vectorization methods. By providing a comprehensively-curated common frame of reference that encompasses all aspects of the French society, the TRF-GIS will help create the conditions for a better understanding of the dramatic socioeconomic changes that France underwent during the seven decades that lasted the Third Republic. Methods General methodology The underlying structure of the TRF-GIS builds on three datasets that are available under open licensing (Figure provides an overview of the construction logic of the TRF-GIS): 1. INSEE's Codes Officiel Géographique (COG) nomenclatures. 18 INSEE's annual COGs provide the official nomenclature of communes and of general administrative constituencies in the geography of a given year. Existing communes are uniquely identified through a five-digit coding scheme, which first two digits correspond to a commune's département and last three to its number within the département, generally assigned in ascending alphabetical order. For instance, the commune of Allonne in the département of Oise (60) holds INSE...

show abstract

Transparency, Reproducibility, and the Credibility of Economics Research

Cited by 299 publications

References 339 publications

Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme

Are replication rates the same across academic fields? Community forecasts from the DARPA SCORE programme

How to Make a Pie: Reproducible Research for Empirical Economics and Econometrics

Mapping the Third Republic. A Geographic Information System of France (1870–1940).

Contact Info

Product

Resources

About