Identifying rainfall-runoff events in discharge time series: a data-driven method based on information theory

Abstract

Abstract. In this study, we propose a data-driven approach for automatically identifying rainfall-runoff events in discharge time series. The core of the concept is to construct and apply discrete multivariate probability distributions to obtain probabilistic predictions of each time step that is part of an event. The approach permits any data to serve as predictors, and it is non-parametric in the sense that it can handle any kind of relation between the predictor(s) and the target. Each choice of a particular predictor data set is equivalent to formulating a model hypothesis. Among competing models, the best is found by comparing their predictive power in a training data set with user-classified events. For evaluation, we use measures from information theory such as Shannon entropy and conditional entropy to select the best predictors and models and, additionally, measure the risk of overfitting via cross entropy and Kullback–Leibler divergence. As all these measures are expressed in “bit”, we can combine them to identify models with the best tradeoff between predictive power and robustness given the available data. We applied the method to data from the Dornbirner Ach catchment in Austria, distinguishing three different model types: models relying on discharge data, models using both discharge and precipitation data, and recursive models, i.e., models using their own predictions of a previous time step as an additional predictor. In the case study, the additional use of precipitation reduced predictive uncertainty only by a small amount, likely because the information provided by precipitation is already contained in the discharge data. More generally, we found that the robustness of a model quickly dropped with the increase in the number of predictors used (an effect well known as the curse of dimensionality) such that, in the end, the best model was a recursive one applying four predictors (three standard and one recursive): discharge from two distinct time steps, the relative magnitude of discharge compared with all discharge values in a surrounding 65 h time window and event predictions from the previous time step. Applying the model reduced the uncertainty in event classification by 77.8 %, decreasing conditional entropy from 0.516 to 0.114 bits. To assess the quality of the proposed method, its results were binarized and validated through a holdout method and then compared to a physically based approach. The comparison showed similar behavior of both models (both with accuracy near 90 %), and the cross-validation reinforced the quality of the proposed model. Given enough data to build data-driven models, their potential lies in the way they learn and exploit relations between data unconstrained by functional or parametric assumptions and choices. And, beyond that, the use of these models to reproduce a hydrologist's way of identifying rainfall-runoff events is just one of many potential applications.

Quantitative precipitation estimation with weather radar using a data- and information-based approach

“…Quantitative precipitation estimation at high temporal and spatial resolution and in high quality are important prerequisites for many hydrometeorological design and management purposes. Besides rain gauges that have their own limitations (Huff, 1970;Nešpor and Sevruk, 1999;Nystuen, 1999;Yang et al, 1999), weather radar plays an increasingly important role in QPE: among other data sources, radar data have been used for urban hydrology (Thorndahl et al, 2017;Cecinati et al, 2017a;Wang et al, 2015), hydrological analysis and modeling (Bronstert et al, 2017;Rossa et al, 2005), real-time QPE (Germann et al, 2006), rainfall climatology (Overeem et al, 2009), rainfall pattern analysis (Kronenberg et al, 2012;Ruiz-Villanueva et al, 2012) and rainfall frequency analysis (Goudenhoofdt et al, 2017). For a comprehensive overview of radar theory and applications, see Battan (1959a, b), Sauvageot (1992), Doviak and Zrnic (1993), Rinehart (1991), Fabry (2015) or Rauber and Nesbitt (2018).…”

“…In this context, it is the aim of this paper to suggest and apply a framework which would use relationships between data expressed as empirical discrete probability distributions (dpd's), and would measure the strength of relations and remaining uncertainties with measures from information theory. Comparable approaches have been suggested by Sharma and Mehrotra (2014) and Thiesen et al (2019): the former use an information-theoretic approach to formulate prediction models for cases where physical relationships are only weakly known but observational records are abundant; the latter emulate expert-based classification of rainfall-runoff events in hydrological time series by constructing dpd's from large sets of training data.…”

Histogram via entropy reduction (HER): an information-theoretic alternative for geostatistics

“…The source code for an implementation of HER, containing spatial characterization, convex optimization, and distribution prediction, is published alongside this paper at https://github. com/KIT-HYD/HER (Thiesen et al, 2020). The repository also includes scripts for exemplifying the use of the functions and the data set used in the case study.…”

“…S. Thiesen et al: HER: an information-theoretic alternative for geostatistics 2016; Thiesen et al, 2019;Mälicke et al, 2020), quantifying uncertainty and evaluating model performance (Chapman, 1986;Liu et al, 2016;Thiesen et al, 2019), estimating information flow (Weijs, 2011;Darscheid, 2017), and measuring similarity, quantity, and quality of information in hydrological models (Nearing and Gupta, 2017;Loritz et al, 2018Loritz et al, , 2019. In the spatial context, information-theoretic measures were used to obtain longitudinal profiles of rivers (Leopold and Langbein, 1962), to solve problems of spatial aggregation and quantify information gain, loss, and redundancy (Batty, 1974;Singh, 2013), to analyze spatiotemporal variability (Mishra et al, 2009;Brunsell, 2010), to address risk of landslides (Roodposhti et al, 2016), and to assess spatial dissimilarity (Naimi, 2015), complexity (Pham, 2010), uncertainty (Wellmann, 2013), and heterogeneity (Bianchi and Pedretti, 2018).…”

On the Automation of Flood Event Separation From Continuous Time Series

“…The power of ML algorithms to detect patterns and to reproduce them in further application could be a solution to this topic. Thiesen et al (2019) demonstrated that data-driven approaches with different predicors can be applied to the task of hydrograph separation. They found that models using discharge as predictors returned the best results.…”