Improved validation framework and R-package for artificial neural network models

Humphrey, Greer B.; Maier, Holger R.; Wu, Wenyan; Mount, Nick J.; Dandy, Graeme C.; Abrahart, Robert J.; Dawson, Christian W.

doi:10.1016/j.envsoft.2017.01.023

Cited by 58 publications

(35 citation statements)

References 95 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Regardless of which model type is used, it is generally considered good practice to conduct an independent assessment of the performance of a model, using data that were not used for model development (e.g., Biondi et al, ; Humphrey et al, ; Power, ). This practice is also commonly used to perform comparative evaluations of the performance of different types of models (i.e., performance on the independent evaluation data is used to infer whether the performance of a particular model can be considered to be superior to that of another; e.g., Dibike & Coulibaly, ; Humphrey et al, ; Li et al, ; Valipour et al, ).…”

Section: Introductionmentioning

confidence: 99%

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data‐Driven Models

Zheng

Maier

et al. 2018

Water Resources Research

Self Cite

View full text Add to dashboard Cite

Hydrological models are used for a wide variety of engineering purposes, including streamflow forecasting and flood‐risk estimation. To develop such models, it is common to allocate the available data to calibration and evaluation data subsets. Surprisingly, the issue of how this allocation can affect model evaluation performance has been largely ignored in the research literature. This paper discusses the evaluation performance bias that can arise from how available data are allocated to calibration and evaluation subsets. As a first step to assessing this issue in a statistically rigorous fashion, we present a comprehensive investigation of the influence of data allocation on the development of data‐driven artificial neural network (ANN) models of streamflow. Four well‐known formal data splitting methods are applied to 754 catchments from Australia and the U.S. to develop 902,483 ANN models. Results clearly show that the choice of the method used for data allocation has a significant impact on model performance, particularly for runoff data that are more highly skewed, highlighting the importance of considering the impact of data splitting when developing hydrological models. The statistical behavior of the data splitting methods investigated is discussed and guidance is offered on the selection of the most appropriate data splitting methods to achieve representative evaluation performance for streamflow data with different statistical properties. Although our results are obtained for data‐driven models, they highlight the fact that this issue is likely to have a significant impact on all types of hydrological models, especially conceptual rainfall‐runoff models.

show abstract

Section: Introductionmentioning

confidence: 99%

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data‐Driven Models

Zheng

Maier

et al. 2018

Water Resources Research

Self Cite

View full text Add to dashboard Cite

show abstract

“…The convergence of the weights of the identifier and controller are shown in Figure 9, respectively, where all of the INN weights converge after 20 s. Calculated by the IPC control station calculation cycle Ts = 100 ms, equivalent to the online iterative training nearly 200 times. The number of iterations is similar to that of some online learning research [30][31][32][33][34].…”

Section: Implementation Of the Drnn Control Algorithmmentioning

confidence: 97%

A New Method of Applying Data Engine Technology to Realize Neural Network Control

Zheng

Song

2019

Algorithms

View full text Add to dashboard Cite

This paper presents a novel diagonal recurrent neural network hybrid controller based on the shared memory of real-time database structure. The controller uses Data Engine (DE) technology, through the establishment of a unified and standardized software architecture and real-time database in different control stations, effectively solves many problems caused by technical standard, communication protocol, and programming language in actual industrial application: the advanced control algorithm and control system co-debugging difficulties, algorithm implementation and update inefficiency, and high development and operation and maintenance costs effectively fill the current technical gap. More importantly, the control algorithm development uses a unified visual graphics configuration programming environment, effectively solving the problem of integrated control of heterogeneous devices; and has the advantages of intuitive configuration and transparent data processing process, reducing the difficulty of the advanced control algorithms debugging in engineering applications. In this paper, the application of a neural network hybrid controller based on DE in motor speed measurement and control system shows that the system has excellent control characteristics and anti-disturbance ability, and provides an integrated method for neural network control algorithm in a practical industrial control system, which is the major contribution of this article.

show abstract

“…Concerning the testing procedure, while the available metrics for the assessment of the forecast quality are a lot, most of the studies use only a few (Krause et al 2005), understating the importance of the testing process despite relevant suggestions (e.g. Humphrey et al 2017). Similarly, the number of the compared forecasting methods is usually small, although benchmarks are commonly included in the relevant comparisons (Pappenberger et al 2015).…”

Section: The Broader Perspectivementioning

confidence: 99%

Comparison of Stochastic and Machine Learning Methods for Multi-Step Ahead Forecasting of Hydrological Processes

Papacharalampous¹,

Tyralis²,

Koutsoyiannis³

2017

Preprint

View full text Add to dashboard Cite

Abstract:We perform an extensive comparison between 11 stochastic to 9 machine learning methods regarding their multi-step ahead forecasting properties by conducting 12 large-scale computational experiments. Each of these experiments uses 2 000 time series generated by linear stationary stochastic processes. We conduct each simulation experiment twice; the first time using time series of 110 values and the second time using time series of 310 values. Additionally, we conduct 92 real-world case studies using mean monthly time series of streamflow and particularly focus on one of them to reinforce the findings and highlight important facts. We quantify the performance of the methods using 18 metrics. The results indicate that the machine learning methods do not differ dramatically from the stochastic, while none of the methods under comparison is uniformly better or worse than the rest. However, there are methods that are regularly better or worse than others according to specific metrics.

show abstract

Improved validation framework and R-package for artificial neural network models

Cited by 58 publications

References 95 publications

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data‐Driven Models

On Lack of Robustness in Hydrological Model Development Due to Absence of Guidelines for Selecting Calibration and Evaluation Data: Demonstration for Data‐Driven Models

A New Method of Applying Data Engine Technology to Realize Neural Network Control

Comparison of Stochastic and Machine Learning Methods for Multi-Step Ahead Forecasting of Hydrological Processes

Contact Info

Product

Resources

About