Adaptive clustering: A method to analyze dynamical similarity and to reduce redundancies in distributed (hydrological) modeling

Abstract

Abstract. In this paper we propose adaptive clustering as a new way to analyse hydrological systems and to reduce computational efforts of distributed modelling, by dynamically identifying similar model elements, clustering them and inferring dynamics from just a few representatives per cluster. It is based on the observation that while hydrological systems generally exhibit large spatial variability of their properties, requiring distributed approaches for analysis and modelling, there is also redundancy, i.e. there exist typical and recurrent combinations of properties, such that sub systems exist with similar properties, which will exhibit similar internal dynamics and produce similar output when in similar initial states and when exposed to similar forcing. Being dependent on all these factors, similarity is hence a dynamical rather than a static phenomenon, and it is not necessarily a function of spatial proximity. We explain and demonstrate adaptive clustering at the example of a conceptual, yet realistic and distributed hydrological model, fit to the Attert basin in Luxembourg by multi-variate calibration. Based on normalized and binned transformations of model states and fluxes, we first calculated time series of Shannon information entropy to measure dynamical similarity (or redundancy) among sub systems. This revealed that indeed high redundancy exists, that its magnitude differs among variables, that it varies with time, and that for the Attert basin the spatial patterns of similarity are mainly controlled by geology and precipitation. Based on these findings, we integrated adaptive clustering into the hydrological model. It constitutes a shell around the model hydrological process core and comprises: Clustering of model elements, choice of cluster representatives, mapping of results from representatives to recipients, comparison of clusterings over time to decide when re-clustering is advisable. Adaptive clustering, compared to a standard, full-resolution model run used as a virtual reality truth, reduced computation time to one fourth, when accepting a decrease of modelling quality, expressed as Nash–Sutcliffe efficiency of sub catchment runoff, from 1 to 0.84. We suggest that adaptive clustering is a promising tool for both system analysis, and for reducing computation times of distributed models, thus facilitating applications to larger systems and/or longer periods of time. We demonstrate the potential of adaptive clustering at the example of a hydrological system and model, but it should apply to a wide range of systems and models across the earth system sciences. Being dynamical, it goes beyond existing static methods used to increase model performance, such as lumping, and it is compatible with existing dynamical methods such as adaptive time-stepping or adaptive gridding. Unlike the latter, adaptive clustering does not require adjacency of the sub systems to be joined.

The role and value of distributed precipitation data for hydrological models

“…time-variant spatial model resolution (Ehret et al 2020). The basic idea of adaptive clustering has been motivated within the work of Zehe et al (2014) who stated that functional similarity in a catchment (or in a model) can emerge if different sub-units are structurally similar (e.g.…”

“…A different avenue to implement a dynamically changing model resolution is adaptive clustering, as recently demonstrated for a spatially distributed conceptual (top-down) model by Ehret et al (2020). This concept allows 30 for continuous hydrological simulations, which use a higher spatial model resolution only at those time steps when it is necessary.…”

“…While the idea of adaptive clustering is promising as it allows a minimum adequate representations of the spatial variability of a hydrological landscape, it has to our knowledge so far only been tested within a simple top-down model (Ehret et al (2020)). It is thus of interest whether such a dynamic clustering is also feasible when using a physically based (bottom-up) model particularly as these models were specifically introduced to explore how system characteristics and driving gradients control hydrological dynamics (Freeze and Harlan, 1969).…”

“…Here we 15 will hence test and develop an adaptive clustering approach using straightforward physical reasoning and implement it into a distributed bottom-up model. The underlying objective is to exploit the value of adaptive clustering as a tool to better understand the temporal relevance of distributed precipitation for the runoff generation of a meso-scale catchment and as by-product reiterate that adaptive clustering could potentially be used to reduce computational times as already discussed in detail by Ehret et al (2020). High computational times are thereby 20 still one of the many reasons why bottom-up are rarely used on larger scales in an spatial explicit manner (Clark et al, 2017).…”

“…Hydrology since at least Sivapalan et al (1987) where they introduced the concept of representative elementary areas. The main novelty of adaptive clustering is that hydrological similar model elements are dynamically grouped and split in the runtime of the model instead of running a constant number of functional similar elements for the entire simulation period (Ehret et al, 2020).…”