Verifying the Solution from a Nonlinear Solver: A Case Study

McCullough, B. D.; Vinod, Hrishikesh D.

doi:10.1257/000282803322157133

Cited by 193 publications

(112 citation statements)

References 26 publications

Supporting

Mentioning

110

Contrasting

Order By: Relevance

“…Despite the fact that previous research (Stokes 2004) had pointed out issues to reproduce even top contributions of the profession that involve non-linear solvers, McCullough and Vinod (2003) does not find a single author who reports some kind of efforts to verify results produced by statistical software that involves non-linear solvers. Consequently McCullough and Vinod (2003) claims that policies yet alone are not sufficient and only both, data and code archives can ensure reproducibility of empirical economic research. When reproduction problems occur at a later stage solid code and data archives are mandatory to trace back results and circumstances of the data generating process.…”

Section: Motivationmentioning

confidence: 96%

“…Against the background of the ongoing development in reproducible research (Koenker and Zeileis 2009) the ability to trace data back to their provider and to describe data extensively becomes a central aspect of an empirical researcher's work 3 . McCullough and Vinod (2003) check five years of American Economic Review articles that involve non-linear solvers for their authors' efforts to verify solutions provided by their respective software package. Despite the fact that previous research (Stokes 2004) had pointed out issues to reproduce even top contributions of the profession that involve non-linear solvers, McCullough and Vinod (2003) does not find a single author who reports some kind of efforts to verify results produced by statistical software that involves non-linear solvers.…”

Section: Motivationmentioning

confidence: 99%

“…McCullough and Vinod (2003) check five years of American Economic Review articles that involve non-linear solvers for their authors' efforts to verify solutions provided by their respective software package. Despite the fact that previous research (Stokes 2004) had pointed out issues to reproduce even top contributions of the profession that involve non-linear solvers, McCullough and Vinod (2003) does not find a single author who reports some kind of efforts to verify results produced by statistical software that involves non-linear solvers. Consequently McCullough and Vinod (2003) claims that policies yet alone are not sufficient and only both, data and code archives can ensure reproducibility of empirical economic research.…”

Section: Motivationmentioning

confidence: 99%

See 2 more Smart Citations

timeseriesdb: Manage and Archive Time Series Data in Establishment Statistics with R and PostgreSQL

Bannert

2015

SSRN Journal

View full text Add to dashboard Cite

Standard-Nutzungsbedingungen:Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden.Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen.Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in KOF Swiss Economic Institute, ETH ZurichAbstract timeseriesdb is an R package which suggests a PostgreSQL database structure to store time series alongside extensive multi-lingual meta information and provides an R database interface including a web based GUI. The timeseriesdb package was designed to handle time series in establishment statistics. Information such as the GDP or data stemming from the aggregation of economic surveys is typically published on a monthly, quarterly or yearly basis. Hence the package is optimized to handle a large amount of different time series as opposed to managing a smaller number of high frequency time series such as real time data obtained from measuring devices. The particular focus of timeseriesdb is to help the user find and extract a particular set of information within a larger set of information. The timeseriesdb package intends to provide the infrastructure for a time series catalog as opposed to handling time series operations on database level. The underlying structure relies on PostgreSQL's hstore data type which allows to store an array of key-value pairs in a single cell. The hstore data type is not only used to reduce the number of records by storing an entire time series in a single record but also to store a record specific amount of multi-lingual meta information items flexibly.

show abstract

Section: Motivationmentioning

confidence: 96%

Section: Motivationmentioning

confidence: 99%

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

timeseriesdb: Manage and Archive Time Series Data in Establishment Statistics with R and PostgreSQL

Bannert

2015

SSRN Journal

View full text Add to dashboard Cite

show abstract

“…Publications such as McCullough and Vinod (2003) demonstrate that in the absence of archives reproducibility is more an exception than the rule. Consequently, many leading economics journals recently introduced mandatory data archives, sometimes even mandatory code archives, coupled with corresponding editorial policies in order to provide sufficient information to facilitate replication.…”

Section: Introductionmentioning

confidence: 99%

Reproducible Econometric Simulations

Kleiber

Zeileis

2013

Journal of Econometric Methods

View full text Add to dashboard Cite

Reproducibility of economic research has attracted considerable attention in recent years. So far, the discussion has focused mainly on reproducibility of empirical analyses. This paper addresses a further aspect of reproducibility, the reproducibility of computational experiments. More specifically, we contribute to the emerging literature on reproducibility in economics along three lines: (i) we document how simulations of various types are an integral part of publications in modern econometrics, (ii) we provide some general guidelines about how to set up reproducible simulation experiments, and, finally, (iii) we provide a case study from time series econometrics that illustrates the main issues arising in connection with reproducibility, emphasizing the use of modular tools.

show abstract

“…On the other hand, the accuracy of software cannot be taken for granted and it is possible to nd serious errors in commonly used statistical programs as shown by surveys such as McCullough (1999), McCullough (2004), and Keeling and Pavur (2007). It is also documented by Newbold et al (1994), McCullough and Vinod (2003), and Stokes (2004) that dissimilar sets of results can be obtained when the same data and model is analyzed through dierent econometric programs. It is evident that researchers and practitioners should treat computer output with a degree of circumspection and verify computer generated forecasts manually or by using more than one software package where available.…”

Section: Introductionmentioning

confidence: 99%

On the importance of verifying forecasting results

Yalta

Jenal²

2009

International Journal of Forecasting

View full text Add to dashboard Cite

We discuss the various sources of error in numerical computations with the use of examples from the literature relevant to time series analysis. We also submit a case where, by manual verication, we were able to discover a plausible forecast to be erroneous due to a number of software aws in the XLSTAT addin for Microsoft Excel. Furthermore, after discussing the alternative techniques for implementing on a computer the ARIMA (AutoRegressive, Integrated, Moving Average) methodology, we show that dierent approaches can cause considerable discrepancies in the results across dierent programs and even within a single software system.

show abstract

Verifying the Solution from a Nonlinear Solver: A Case Study

Cited by 193 publications

References 26 publications

timeseriesdb: Manage and Archive Time Series Data in Establishment Statistics with R and PostgreSQL

timeseriesdb: Manage and Archive Time Series Data in Establishment Statistics with R and PostgreSQL

Reproducible Econometric Simulations

On the importance of verifying forecasting results

Contact Info

Product

Resources

About