Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening

Abstract

This paper revisits the instrumental variable (IV) approach in Lipscomb et al. (2013, 2021) to study the impacts of electrification. We first identify errors in the construction of the dataset, including the modelled IV. Revised estimates on main outcomes and mechanisms are statistically insignificant, with substantially lower effect sizes. We second develop a framework that accounts for weak IVs and discourages specification screening. We find that most theoretically justified specifications in the Lipscomb et al. (2013, 2021) data deliver insignificant results. The proposed framework is transferable to other IV applications to reduce potential bias stemming from researcher’s or replicator’s discretion.

Rural electrification, the credibility revolution, and the limits of evidence-based policy

“…Another reason to be concerned is that these geographical IVs are often weak IVs, which is not a problem per se if appropriate remedies are used. But these remedies are less effective if weakness concurs with violated exclusion restrictions (Bensch et al 2020) and if scholars screen specifications based on first-stage strength (Ankel- Peters et al 2023a). Related to the screening aspect, IVs are suspected of being more prone to publication bias and p-hacking (Brodeur et al 2020), because "when using a non-experimental method like IV there are many points at which a researcher exercises discretion in ways that could affect statistical significance".…”