The need for impact evaluation in electricity access research

Abstract

Universal household electrification is a key component of the United Nations Sustainable Development Goals, but the evidence base for social and economic impacts of electricity access remains unclear. Here we report results from a systematic review of impact evaluations of household electrification based on five key outcome measures. We only find 31 studies that conduct statistical hypothesis tests to assess impacts. Among these, seven draw on a randomized experiment designed for causal inference. The randomized experimental studies generate fewer positive results than observational or quasi-experimental studies, such as correlational, instrumental variable, and difference-indifferences designs. These results call for a reassessment of what we know about the impacts of household electrification. They also call for major investment in impact evaluation of electricity access using randomized controlled trials, with a particular focus on when and how energy access interventions can furnish large benefits to their intended beneficiaries. Large-scale impact evaluations using experimental methods will require close collaboration between policymakers and researchers.

Does Household Electrification Supercharge Economic Development?

“…Note that we do not conduct a comprehensive literature review, given that there is already excellent work along these lines. For example, see Bayer et al (2019) for a systematic review; van de Walle et al (2015) for a general literature review; Morrissey (2018) for a discussion on productive uses of electric power; Peters and Sievert (2016) for a discussion of the studies using African data; and Bernard (2012) for historical context on electrification initiatives in Sub-Saharan Africa. Instead, we attempt to fill a gap in previous reviews by discussing why the existing set of studies might reach such different conclusions, focusing on differences in econometric methods, the types of electrification interventions studied, the potential for spillovers, and differences in regions and populations.…”

Shades of darkness or light? A systematic review of geographic bias in impact evaluations of electricity access

“…Surely, the number of studies that pass the filters to end up in our sample is small. However, because the small number of studies that use statistical hypothesis testing to assess impacts for household electricity access is the result of a transparent and replicable systematic review of the literature this finding is itself an important insight for electricity access scholarship (Bayer et al 2019). One important message of our paper here is therefore that very few rigorous impact evaluations of electricity access exist.…”

“…Going beyond traditional before-and-after analysis, an impact evaluation typically compares the effect of a policy on a 'treated' group against a 'control' group (Waddington et al 2018). So far, few impact evaluations have focused on electricity access (Bernard 2012), and experimental research designs that would allow for more robust inference for policy development are still scarce (Bayer et al 2019).…”

“…Our systematic review however goes beyond these existing works by documenting regional clustering in electricity access impact evaluations across all developing countries around the world and by translating how regional clustering affects the assessed outcomes and impacts. While others show that the chosen method (e.g., observational versus experimental methods) matters for impact assessment (Bayer et al 2019), this paper focuses on understanding the extent of geographical bias in the electricity access literature. This is key for building an effective body of scholarship to guide policy development.…”

Satellite Observations Reveal Inequalities in the Progress and Effectiveness of Recent Electrification in Sub-Saharan Africa

“…These results suggest that, even under a scenario where universal access in terms of availability of electricity supply is achieved, inequalities may persist, undermining the achievement of several of the SDGs, and potentially driving internal migration. Recent literature [69][70][71][72][73][74] has highlighted how dimensions other than physical access to electricity, such as reliability, have important impacts on the benefits of access, particularly for small and medium businesses that drive much of the growth in developing countries. 52,75 The use of lightintensity data to derive metrics related to the electricity access tier, as done here, can thus also illuminate important qualitative dimensions of electricity access.…”