The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.

No abstract

Impact evaluation (IE) of large infrastructure presents numerous challenges, and investments in urban piped water and sanitation are no exception. Here we present methods for more systematic assessment of the implications of such interventions, discussing tradeoffs between validity, relevance and practicality that arise from alternative approaches. Then, to more clearly illustrate the many issues that typically arise in such IEs, we draw on an example application in Zarqa, Jordan, where the Millennium Challenge Corporation invested about US$275 million to upgrade and extend piped water and sewer networks, as well as increase the capacity of the country's largest wastewater treatment plant. The theory of change for the intervention took a systems view of impacts: the project aimed to improve water supply to urban areas while maintaining flows to irrigators through enhanced wastewater reuse. The case adds valuable evidence on the impacts of large infrastructure investments and illustrates well the challenges of capturing spillovers, mitigating study contamination, maintaining statistical power, and determining overall welfare effects, in situations involving diverse market and nonmarket impacts. These limitations notwithstanding, the case highlights the high value of conducting IEs, and why applied researchers should not give up on pragmatic and interdisciplinary collaborations to evaluation in the face of complex interventions.

Impact evaluation (IE) of large infrastructure presents numerous challenges, and investments in urban piped water and sanitation are no exception. Here we present methods for more systematic assessment of the implications of such interventions, discussing tradeoffs between validity, relevance and practicality that arise from alternative approaches. Then, to more clearly illustrate the many issues that typically arise in such IEs, we draw on an example application in Zarqa, Jordan, where the Millennium Challenge Corporation invested about US$275 million to upgrade and extend piped water and sewer networks, as well as increase the capacity of the country's largest wastewater treatment plant. The theory of change for the intervention took a systems view of impacts: the project aimed to improve water supply to urban areas while maintaining flows to irrigators through enhanced wastewater reuse. The case adds valuable evidence on the impacts of large infrastructure investments and illustrates well the challenges of capturing spillovers, mitigating study contamination, maintaining statistical power, and determining overall welfare effects, in situations involving diverse market and nonmarket impacts. These limitations notwithstanding, the case highlights the high value of conducting IEs, and why applied researchers should not give up on pragmatic and interdisciplinary collaborations to evaluation in the face of complex interventions.

As global population and consumption of water rise, concerns that humankind is entering a new age of global water scarcity are increasingly widespread (Liu et al., 2017). To some, this rising water scarcity is worrisome because water is uniquely essential for myriad purposes-drinking and critical domestic uses, as an input to food and industrial production, and for general human and ecological well-being (Hanemann, 2006). Some predict that water's essentialness will inevitably lead to a zero sum game and loss of livelihoods for specific users, widespread social destabilization, and environmental damage (Joy et al., 2020;Postel, 2000). Such warnings are perhaps most commonly heard in countries or regions where water scarcity is becoming a binding constraint, for example, the Middle East, Western United States, parts of Australia, and in river basins with intense water competition (e.g., the upstream Ganges, Nile, Mekong, or Tigris-Euphrates). Indeed, much of the globe already experiences acute economic water scarcity, due to a lack of high quality infrastructure and an inability of institutions to consistently provide the resource to end users (Rijsberman, 2006). However, both institutional and infrastructure solutions, when designed and operated effectively, can dramatically improve water management, and thereby ease tensions.At the same time, the effects of both water infrastructure and management interventions often vary, and need to be understood within their particular contexts. Learning which interventions work, and under what conditions, is vitally important, both for the very practical work of informing subsequent context-specific interventions, and for applying broader lessons about drivers and impediments of key mechanisms of change to other contexts. Still, the methods to learn about impacts and mechanisms in the water sector-particularly the science of impact evaluation (IE) as applied to water and sanitation projects-remain imperfect. In short, the fundamental challenge facing researchers working on such IEs is to isolate causal effects of infrastructure from the many other contemporaneous changes that affect water supply and sanitation services, water resources systems, and human welfare and well-being. In addition, as we discuss in this paper, the typical "gold standard" methodology for determining causal impacts, the randomized controlled trial (RCT) (