Reproducible research in computational economics: guidelines, integrated approaches, and open source software

Abstract

Traditionally, computer and software applications have been used by economists to off-load otherwise complex or tedious tasks onto technology, freeing up time and intellect to address other, intellectually more rewarding, aspects of research. On the negative side, this increasing dependence on computers has resulted in research that has become increasingly difficult to replicate. In this paper, we propose some basic standards to improve the production and reporting of computational results in economics for the purpose of accuracy and reproducibility. In particular, we make recommendations on four aspects of the process: computational practice, published reporting, supporting documentation, and visualization. Also, we reflect on current developments in the practice of computing and visualization, such as integrated dynamic electronic documents, distributed computing systems, open source software, and their potential usefulness in making computational and empirical research in economics more easily reproducible. Copyright Springer Science+Business Media, LLC 2007Economic methodology, Econometric software, Other computer software, B4, C87, C88,

On reproducible econometric research

“…A historical survey of the early development of econometric software may be found in Renfro (2004), see also Ooms and Doornik (2006). There is also an extensive literature on the individual merits and comparative performance of various forms of statistical software; see Baiocchi (2007) for further references, and a more proscriptive review of reproducibility issues. Here, we will try to restrict our attention quite narrowly to properties of programming environments that might facilitate reproducible research.…”

How to Make a Pie: Reproducible Research for Empirical Economics and Econometrics

“…There exist a wide range of technologies, from basic to more complex systems, that can be implemented to handle these electronic documents. Text files (such as Readme files) can be a simple and efficient way of documenting any element used in a task, and even the task itself (Baiocchi, 2007;Dupas & Robinson, 2013). 18 Other simple tools involve lists and spreadsheets to keep track of ideas and record notes and information about tasks.…”

Establishing a framework for Open Geographic Information science

“…Given these imperatives, improving methods of reproducible research is a topic gaining traction across a range of scientific disciplines (Stodden 2014), with examples arising, but not exclusive to bioinformatics (Gentleman andTemple Lang 2004, McMurdie andHolmes 2015), signal processing (Vandewalle et al 2009), gene pattern analysis (Reich et al 2006), acoustics (Kovacevic 2007), epidemiology (Peng et al 2006) and economics (Baiocchi 2007). More generally, within both science and social science, there are a growing number of initiatives that aim to explicitly test the reproducibility of peer-reviewed research, for example, through the 'crowd sourced' Reproducibility Project (openscienceframework.…”