Improving hydrologic models for predictions and process understanding using neural ODEs

Abstract

Abstract. Deep learning methods have frequently outperformed conceptual hydrologic models in rainfall-runoff modelling. Attempts of investigating such deep learning models internally are being made, but the traceability of model states and processes and their interrelations to model input and output is not yet fully understood. Direct interpretability of mechanistic processes has always been considered an asset of conceptual models that helps to gain system understanding aside of predictability. We introduce hydrologic neural ordinary differential equation (ODE) models that perform as well as state-of-the-art deep learning methods in stream flow prediction while maintaining the ease of interpretability of conceptual hydrologic models. In neural ODEs, internal processes that are represented in differential equations, are substituted by neural networks. Therefore, neural ODE models enable the fusion of deep learning with mechanistic modelling. We demonstrate the basin-specific predictive performance for 569 catchments of the continental United States. For exemplary basins, we analyse the dynamics of states and processes learned by the model-internal neural networks. Finally, we discuss the potential of neural ODE models in hydrology.

Long‐Lead Drought Forecasting Across the Continental United States Using Burg Entropy Spectral Analysis With a Multiresolution Approach

“…BESA also, in nature, is interpretable how they forecast scPDSI. LSTM or DL can also be improved in terms of forecast accuracies and interpretability when they are internally or externally coupled with physics or prior knowledge in the rim of TGDS (Höge et al., 2022; Jiang et al., 2020; Karpatne et al., 2017; Rackauckas et al., 2020). On the other hand, DL can extract data features that the conventional hydrologic approach can not use.…”

“…However, without predefined structures governing a problem, ML models require extensive homogeneous training data and might lose their reliability under changing conditions (e.g., climate change), especially when “big data which can address dynamics” is not always available (Nearing et al., 2021; Rackauckas et al., 2020). Furthermore, although the interpretability of deep learning (DL) has improved with sensitivity analysis and integrated gradients and expected gradients methods that can facilitate the understanding of feature importance in a learning process (Jiang et al., 2020; Kratzert et al., 2019; Nearing et al., 2021; Samek et al., 2019; Sundararajan et al., 2017), those methods for interpretability have their own assumptions and/or limitations (Höge et al., 2022; Sundararajan et al., 2017). Therefore, “theory‐guided data science (TGDS)” and generalizable models with scientific consistency and interpretability are needed (Karpatne et al., 2017; Nearing et al., 2021).…”

Using a long short-term memory (LSTM) neural network to boost river streamflow forecasts over the western United States

“…Future work should look at the contribution made by the LIS-FLOOD streamflow input data to the final LSTM forecasts and whether the impact is significant enough to warrant the additional resources. Alternatively, future research could investigate incorporating machine learning techniques directly into LISFLOOD either to speed up calculations (Höge et al, 2022;Rackauckas et al, 2020;Raissi et al, 2020) or to repli-cate processes that are not currently modelled in LISFLOOD within GloFAS such as the impact of reservoir management.…”

“…However, Frame et al (2022) found that the addition of the mass constraint reduced the skill when predicting extreme values suggesting further work is necessary. Alternatively, other studies have used neural networks in process-based models to solve differential equations more efficiently whilst allowing interpretability of the output (Höge et al, 2022;Rackauckas et al, 2020;Raissi et al, 2020).…”

Distributed Hydrological Modeling With Physics‐Encoded Deep Learning: A General Framework and Its Application in the Amazon

“…Bhasme et al (2022) used water storages simulated by a physics-based model as the inputs for a DL streamflow model, improving the DL model's physical consistency. Physical knowledge can also be directly encoded into a DL model (M. Chen et al, 2023;Höge et al, 2022;Kraft et al, 2022;Reichstein et al, 2019). For example, Jiang et al (2020) wrapped the physical processes depicted by a conceptual hydrological model (i.e., EXP-HYDRO) into a recurrent neural network (RNN).…”