Bootstrap Aggregation and Cross‐Validation Methods to Reduce Overfitting in Reservoir Control Policy Search

Abstract

Policy search methods provide a heuristic mapping between observations and decisions and have been widely used in reservoir control studies. However, recent studies have observed a tendency for policy search methods to overfit to the hydrologic data used in training, particularly the sequence of flood and drought events. This technical note develops an extension of bootstrap aggregation (bagging) and cross‐validation techniques, inspired by the machine learning literature, to improve reservoir control policy performance on out‐of‐sample hydrological sequences. We explore these methods using a case study of Folsom Reservoir, California, using control policies structured as binary trees, and streamflow resampling based on the paleo‐inflow record. Results show that calibration‐validation strategies for policy selection coupled with certain ensemble aggregation methods can improve out‐of‐sample performance in water supply and flood risk objectives over baseline performance given fixed computational costs. Our findings highlight the potential to improve policy search methodologies by leveraging these well‐established model training strategies from machine learning.

A State‐of‐the‐Art Review of Optimal Reservoir Control for Managing Conflicting Demands in a Changing World

“…This approach can overfit the policy parameters to the particular stochastic realizations experienced during the simulation‐based optimization, potentially yielding impressive calibration results that can largely degrade when tested on out‐of‐sample observations. This issue can be overcome by splitting the available observations (or an ensemble of synthetically generated records) into two statistically equivalent data sets to optimize the policy parameters on the first set of data and test policy performance on the second data set (Brodeur et al., 2020). Another often overlooked aspect of DPS is the a priori definition of the policy architecture, generally based on intuition, analytical methods, or on few trial‐and‐error experiments.…”

Bagging-based Adaptive Synthetic Oversampling Algorithm for Robust Policy Search in Reservoir Management

“…To mitigate this limitation, previous studies have been employed to optimize control policies by using sufficient training sequences from synthetically generated scenarios (de la Cruz Courtois et al, 2021;Salazar et al, 2017;Tsoukalas and Makropoulos, 2015). Other works have adopted an additional sequence to improve optimized control policies over other synthetic scenarios not used in training (Brodeur et al, 2020;Quinn et al, 2017).…”

A Multivariate Approach to Generate Synthetic Short‐To‐Medium Range Hydro‐Meteorological Forecasts Across Locations, Variables, and Lead Times

“…This limitation extends to hydrologic forecasts forced by those climate hindcasts (Demargne et al., 2014). Thus, a fairly short time period of available hindcasts (at most ∼40 years) must be parsed into even smaller periods to enable calibration and testing of policies, creating the potential for overfitting and poor out‐of‐sample performance (Brodeur et al., 2020; Herman et al., 2020; Nayak et al., 2018).…”

“…This limitation extends to hydrologic forecasts forced by those climate hindcasts (Demargne et al, 2014). Thus, a fairly short time period of available hindcasts (at most ∼40 years) must be parsed into even smaller periods to enable calibration and testing of policies, creating the potential for overfitting and poor out-of-sample performance (Brodeur et al, 2020;Herman et al, 2020;Nayak et al, 2018).Synthetic forecasts offer a solution to overcome this challenge. Synthetic forecasts are generated by adding random error to observational records, such that the resulting series is statistically indistinguishable from forecasts developed using a physically based model.…”