Forecasting Residential Water Consumption in California: Rethinking Model Selection

Abstract

Urban water managers use forecasts of water consumption to determine management decisions and investment choices. Public reports show that water utilities rely on forecast models that are not selected based on their out-of-sample prediction performance; further, these reports frequently only present a single forecast instead of a range of forecasts. In our review of the academic literature on forecasting long-term water consumption, only a few analyses consider out-of-sample prediction performance measures to assess prediction ability. In none of these analyses did out-of-sample prediction performance drive model selection. Ensemble-type long-term forecasts based on multiple models were also lacking. Using annual data on single-family residential water consumption in Southern California, we show that predictive ability varies drastically depending on how the forecast model is selected. As an illustration of how forecast performance is affected when the criteria for model selection does support forecasting objectives, we compare statistical models with the best in-sample and out-of-sample prediction performance. We find that the models with the best in-sample performance over-estimate consumption five years out by 10%-25% compared to actual consumption. In contrast, the top 1% of models selected based on out-of-sample prediction performance came within 1% of actual consumption. Finally, we compare the performance of our ensemble-type forecasts to those reported in public documents derived from models selected based on non-out-of-sample prediction performance criteria. Our results highlight the benefits of (i) using an out-of-sample evaluation criterion to guide model selection and (ii) reporting ensemble forecasts in lieu of a single forecast. Key Points:• Our forecasting method makes out-of-sample prediction performance the criterion for model selection • Models selected on out-of-sample performance drastically outperform forecast models employed in practice • We predict residential water consumption in 2035 for Southern California, consumption decreases 0.5 percent to over a 10 percent reduction

Comparing Water Use Forecasting Model Selection Criteria: The Case of Commercial, Institutional, and Industrial Sector in Southern California

“…In addition to software packages, regression-based econometric models are currently being used by the utilities for forecast purposes. See Buck, Auffhammer, Soldati, et al [11] for a summary of methods being used by a group of large California utilities; they show out-of-sample performance is not commonly used as a model selection criterion and forecast modelers typically only consider a narrow set of models.…”

“…For a geographical reference, see the map of the region published by the Southern California Association of Governments [63]. The dataset used here is a subset of a larger dataset collected for a study about forecasting single-family residential (SFR) sector water use [11]. Data collection effort, therefore, was focused on the retailers that reported more than 3000 single-family residential accounts as it is estimated that these retailers account for about 99% of this sector.…”

Forecast of regional water demand based on NSGAII-FORAGM

“…The optimal allocation of water resources is an effective means to realize the sustainable development and utilization of water resources (Zhou et al 2015). Accurately predicting the regional water demand is a vital method to realize the optimal allocation of water resources in the region, and it has certain significance for the rational water distribution of the region (Zhai et al 2009;Buck et al 2020). Therefore, the accurate forecasting of regional water demand has been an urgent problem to be solved with the increasing shortage of water resources.…”

Integrated Deep Neural Networks-Based Complex System for Urban Water Management

“…However, the collected real datasets are unlikely to meet statistical assumptions. And gray theory models generally solve the modeling of small sample size and sparse data [33,34]. But water consumption is affected by many factors, and it is more suitable for deep learning methods for such prediction problems with complex data volume and difficult to fit fluctuation patterns [35,36].…”