Rainfed agriculture plays and will continue to play a dominant role in providing food and livelihoods for an increasing world population. We describe the world's semi-arid and dry sub-humid savannah and steppe regions as global hotspots, in terms of water related constraints to food production, high prevalence of malnourishment and poverty, and rapidly increasing food demands. We argue that major water investments in agriculture are required. In these regions yield gaps are large, not due to lack of water per se, but rather due to inefficient management of water, soils, and crops. An assessment of management options indicates that knowledge exists regarding technologies, management systems, and planning methods. A key strategy is to minimise risk for dry spell induced crop failures, which requires an emphasis on water harvesting systems for supplemental irrigation. Large-scale adoption of water harvesting systems will require a paradigm shift in Integrated Water Resource Management (IWRM), in which rainfall is regarded as the entry point for the governance of freshwater, thus incorporating green water resources (sustaining rainfed agriculture and terrestrial ecosystems) and blue water resources (local runoff). The divide between rainfed and irrigated agriculture needs to be reconsidered in favor of a governance, investment, and management paradigm, which considers all water options in agricultural systems. A new focus is needed on the meso-catchment scale, as opposed to the current focus of IWRM on the basin level and the primary focus of agricultural improvements on the farmer's field. We argue that the catchment scale offers the best opportunities for water investments to build resilience in smallscale agricultural systems and to address trade-offs between water for food and other ecosystem functions and services.