Scenario-based fitted Q-iteration for adaptive control of water reservoir systems under uncertainty

Abstract

This study presents a novel approach called scenario-based Fitted Q-Iteration (sFQI) for controlling water reservoir systems under climate uncertainty. In these problems, robust control frameworks, based on worst-case realization, are usually adopted. Yet, these might be overly conservative. In this paper, we use sFQI to design adaptive control policies by enlarging the state space to include the space of the uncertain system's parameters. This allows obtaining a control policy for any scenario in the uncertainty set with a single learning process. The method is demonstrated on a simplified model of the Lake Como system, a regulated lake operated for ensuring reliable water supply to downstream users. Numerical results show that the sFQI algorithm successfully identifies adaptive solutions to operate the system under different inflow scenarios, which outperform the control policy designed under historical conditions. Moreover, the sFQI policy generalizes over inflow scenarios not directly experienced during the policy design, thus alleviating the risk of mis-adaptation, namely the design of a solution fully adapted to a scenario that is different from the one that will actually realize.

Sustainable WEF Nexus Management: A Conceptual Framework to Integrate Models of Social, Economic, Policy, and Institutional Developments

“…In addition to these models, scenarios driven by the global Representative Concentration Pathways (IPCC, 2007) and Shared Socioeconomic Pathways (Kriegler et al, 2012(Kriegler et al, , 2013O'Neill et al, 2014) are used in order to frame potential future trends in terms of climate change, water demand and availability, energy consumption and production, demographic and economic development. These scenarios also seek to address uncertainty within the systems being examined and modeled (Bertoni et al, 2017). Due consideration of uncertainty within the framework is critical, and is characterized firstly in terms of long run uncertainty based on the alternative scenarios for population growth, energy, GDP, climate change, etc.…”

“…In the case of the DAFNE project, this process is embodied within a decision support tool known as the DAF. The DAF (Burlando et al, 2018) screens potential WEF management actions (e.g., Dam construction or reservoir operation policy) under various scenarios, sequencing them in different combinations to form candidate developmental pathways (Bertoni et al, 2017).…”