Legitimising data-driven models: exemplification of a new data-driven mechanistic modelling framework

Abstract

Abstract.In this paper the difficult problem of how to legitimise data-driven hydrological models is addressed using an example of a simple artificial neural network modelling problem. Many data-driven models in hydrology have been criticised for their black-box characteristics, which prohibit adequate understanding of their mechanistic behaviour and restrict their wider heuristic value. In response, presented here is a new generic data-driven mechanistic modelling framework. The framework is significant because it incorporates an evaluation of the legitimacy of a data-driven model's internal modelling mechanism as a core element in the modelling process. The framework's value is demonstrated by two simple artificial neural network river forecasting scenarios. We develop a novel adaptation of first-order partial derivative, relative sensitivity analysis to enable each model's mechanistic legitimacy to be evaluated within the framework. The results demonstrate the limitations of standard, goodness-of-fit validation procedures by highlighting how the internal mechanisms of complex models that produce the best fit scores can have lower mechanistic legitimacy than simpler counterparts whose scores are only slightly inferior.

Data-driven modelling approaches for socio-hydrology: opportunities and challenges within the Panta Rhei Science Plan

“…Olden and Jackson 2002, Kingston et al 2005 and sensitivity analysis of its response function (e.g. Sudheer 2005, Mount et al 2013, Dawson et al 2014. Certain data-driven methods, such as the adaptive neuro-fuzzy inference system (ANFIS) (e.g.…”

“…Where robust, a priori hypotheses about hydrologic processes exist, exploration of the internal behaviour of data-driven, predictive models offers a means by which the model's legitimacy may be assessed by the extent to which its response function behaviour can be rationalized in mechanistic and process terms (e.g. Mount et al 2013). Similarly, disaggregation of the response function into the partial behaviours that are associated with each of the model's structural components can be used to determine whether the model is structured in a rational manner (e.g.…”

“…Nouraini and Fard 2012), considered both one-at-a-time or in combination (i.e. through first-, second-or higher-order analyses), offer an important means by which the characteristics of the relationships between input and output variables can be revealed (Mount et al 2013). These can then be used as heuristic support for hypothesis formulation about the processes and feedbacks by which system responses are generated.…”

“…While a number of methods exist in the literature for achieving the latter (e.g. Olden and Jackson 2002, Sudheer 2005, Kingston et al 2006, Jain and Kumar 2009, Mount et al 2013, Dawson et al 2014, these methods are not adopted widely in practice (see Wu et al 2014). Even in relation to predictive evaluation, greater care needs to be taken that appropriate data-splitting methods are used, so that predictive performance on the 'validation' set is representative of that over the domain of the available data (Bowden et al 2002, Anctil and Lauzon 2004, May et al 2010, Wu et al 2013.…”

Streamflow simulation methods for ungauged and poorly gauged watersheds

“…Abrahart et al (2012) present recent ANN applications and procedures in streamflow modelling and forecasting, which include modular design concepts, ensemble experiments, and hybridization of ANNs with typical hydrological models. Furthermore, ANNs have been used for combining the outputs of different rainfall-runoff models in order to improve the prediction and modelling of streamflow (Shamseldin et al, 1997;Chen and Adams, 2006;Kim et al, 2006;Nilsson et al, 2006;Cerda-Villafana et al, 2008;Liu et al, 2013) and the river flow forecasting (Brath et al, 2002;Shamseldin et al, 2002;Anctil et al, 2004a;Srinivasulu and Jain, 2009;Elshorbagy et al, 2010;Mount et al, 2013).…”