<p>The increasing stress on water resource systems has prompted researchers to look for ways to improve the performance of reservoir operations. Changes in demand, various hydrological inputs, and new environmental stresses are among issues that water managers face. These concerns have sparked interest in applying different techniques to determine reservoir operation policy to improve reservoir system performance. As the resolution of analysis rises, it becomes more difficult to effectively represent a real-world system using currently available approaches for determining the best reservoir operation policy. One of the challenges is the "curse of dimensionality," which occurs when the discretization of the state and action spaces becomes finer or when more state or action variables are taken into account. Because of the dimensionality curse, the number of state-action variables is limited, rendering dynamic programming (DP) and stochastic DP (SDP) ineffective in handling complex reservoir optimization issues. Reinforcement learning (RL) is one way to overcome the aforementioned curses of stochastic optimization of water resources systems. RL is a well-known and influential technique in machine learning research that can solve a wide range of optimization and simulation challenges. In this study, a novel continuous-action deep RL algorithm called Deep Deterministic Policy Gradients (DDPG) is applied to solve the DP problem for the Folsom Reservoir system located in California, US. Without requiring any model simplifications or surrendering any of the critical characteristics of DP, the employed continuous action-space RL method effectively overcomes dimensionality concerns. The system model employs an iterative learning method that takes into account delayed rewards without requiring an explicit probabilistic model of hydrologic processes, and it can learn the best actions that maximize total expected reward by interacting with a simulated environment. This research is funded by the US Geological Survey.</p>

Abstract. Floods are among the most destructive natural hazards in the world, posing numerous risks to societies and economies globally. Accurately understanding and modeling floods driven by extreme rainfall events has long been a challenging task in the domains of hydrologic science and engineering. Unusual catchment responses to flooding cause great difficulty in predicting the variability and magnitude of floods, as well as proposing solutions to manage large volumes of overland flow. The usage of Nature-Based Solutions (NBS) has proved to be effective in the mitigation of flood peak rate and volume in urban or coastal areas, yet it is still not widely implemented due to limited knowledge and testing compared to traditional engineering solutions. This research examined an integrated hydrological and hydraulic modeling system to understand the response of an at-risk watershed system to flooding and evaluate the efficacy of NBS measures. Using the Hydrologic Engineering Center Hydrologic Modeling System and River Analysis System (HEC-HMS and HEC-RAS) software, an integrated hydrologic-hydraulic model was developed for Hurricanes Matthew (2016) and Florence (2018) driven floods across the Little Pee Dee-Lumber Rivers watershed, North and South Carolina (the Carolinas), USA. The focus was on Nichols town, a small town that has been disproportionately impacted by flooding during these two hurricane events. Different NBS measures including flood storage ponds, riparian reforestation, and afforestation in croplands were designed, modeled, and evaluated. Hurricane Matthew's flooding event was used for evaluating the NBS scenarios given its high simulation accuracy in flood inundation compared to the less accurate results obtained for Hurricane Florence. The scenario comparison evidenced that large-scale natural interventions, such as afforestation in croplands, can reduce the inundated area in Nichols town by 8 % to 18 %. On the contrary, the smaller-scale interventions such as riparian reforestation and flood storage ponds showed a negligible effect of only 1 % on flood mitigation.