On the robustness/replication of econometric analyses from nonlinear models using various commonplace software packages

Abstract

Because replicability is an important part of every scientific endeavor, this research deals with comparing and contrasting parameter estimates, standard errors, and p-values from the estimation of five commonly encountered nonlinear models in applied econometrics. Commonplace software packages indigenous to econometrics and statistics are used, namely EVIEWS 11.0, SAS 9.4, Stata 17, and R 4.1.2 in five replication exercises to determine potential differences, if any, in empirical results. The hypothesis that mainstream software packages generate different empirical results in the estimation of nonlinear models is confirmed for the polynomial distributed lag (PDL) model and the GARCH(1,1) model. For the probit model and the Barten synthetic demand system model, the differences in parameter estimates, standard errors, and p-values are less evident across the four commonly used software packages. For the Tobit model, the respective sets of parameter estimates, standard errors, and p-values are nearly identical across the respective software packages. Economic analysts should not just accept estimation results uncritically, but instead, conduct sensitivity analyses involving the use of at least two software packages. The agricultural economics

Replications in agricultural economics

“…While some estimation results are very similar across software packages, the author finds different empirical results when using different software for some models and software packages. However, Capps (2023) concludes that even for these models the differences in the results do not have economic or practical significance. As this may be different for other empirical applications, he recommends that economists could improve robustness of findings by conducting sensitivity analyses using at least two software packages.…”

“…The authors suggest that it takes the whole scientific community (editors, reviewers, and funders) to evoke changes towards more transparency. Capps (2023) investigates the robustness/replicability of five different econometric analyses from nonlinear models using various commonplace software packages, utilizing a series of previously published examples as case studies (three analyses from published articles and two analyses from lecture notes). While some estimation results are very similar across software packages, the author finds different empirical results when using different software for some models and software packages.…”