Comparison of learning-based wastewater flow prediction methodologies for smart sewer management

Karimi, Hazhar Sufi; Natarajan, Balasubramaniam; Ramsey, Chris L.; Henson, Jeffrey; Tedder, Joshua L.; Kemper, Errin

doi:10.1016/j.jhydrol.2019.123977

Cited by 28 publications

(13 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has contributed to their widespread application in hydrological modelling (Abrahart et al, 2012). In some cases, ensembles are not combined, and the collections of predictions are used to estimate the uncertainty associated with the diversity between ensemble members (Tiwari and Chatterjee, 2010;Abrahart et al, 2012). While this approach has obvious advantages, it is not possible for all types of ensembles, such as the boosting methods, which are also used in this research.…”

Section: Ensemble-based Techniquesmentioning

confidence: 99%

Resampling and ensemble techniques for improving ANN-based high-flow forecast accuracy

Snieder

Abogadil

Khan

2021

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

Abstract. Data-driven flow-forecasting models, such as artificial neural networks (ANNs), are increasingly featured in research for their potential use in operational riverine flood warning systems. However, the distributions of observed flow data are imbalanced, resulting in poor prediction accuracy on high flows in terms of both amplitude and timing error. Resampling and ensemble techniques have been shown to improve model performance on imbalanced datasets. However, the efficacy of these methods (individually or combined) has not been explicitly evaluated for improving high-flow forecasts. In this research, we systematically evaluate and compare three resampling methods, random undersampling (RUS), random oversampling (ROS), and the synthetic minority oversampling technique for regression (SMOTER), and four ensemble techniques, randomised weights and biases, bagging, adaptive boosting (AdaBoost), and least-squares boosting (LSBoost), on their ability to improve high stage prediction accuracy using ANNs. These methods are implemented both independently and in combined hybrid techniques, where the resampling methods are embedded within the ensemble methods. This systematic approach for embedding resampling methods is a novel contribution. This research presents the first analysis of the effects of combining these methods on high stage prediction accuracy. Data from two Canadian watersheds (the Bow River in Alberta and the Don River in Ontario), representing distinct hydrological systems, are used as the basis for the comparison of the methods. The models are evaluated on overall performance and on typical and high stage subsets. The results of this research indicate that resampling produces marginal improvements to high stage prediction accuracy, whereas ensemble methods produce more substantial improvements, with or without resampling. Many of the techniques used produced an asymmetric trade-off between typical and high stage performance; reduction of high stage error resulted in disproportionately larger error on a typical stage. The methods proposed in this study highlight the diversity-in-learning concept and help support future studies on adapting ensemble algorithms for resampling. This research contains many of the first instances of such methods for flow forecasting and, moreover, their efficacy in addressing the imbalance problem and heteroscedasticity, which are commonly observed in high-flow and flood-forecasting models.

show abstract

Section: Ensemble-based Techniquesmentioning

confidence: 99%

Resampling and ensemble techniques for improving ANN-based high-flow forecast accuracy

Snieder

Abogadil

Khan

2021

Hydrol. Earth Syst. Sci.

View full text Add to dashboard Cite

show abstract

“…Jang et al [46] employed a deep learning model using LSTM to predict the business failure of the construction market, whose prediction model had superior prediction performance from a long-term perspective when the construction market and macroeconomic variables were used in addition to accounting variables. Karimi et al [47] adopted an LSTM based model to predict the wastewater flow in the smart sewer management. They harbor the idea that LSTM is suitable for prediction when the dynamic of the system is changing.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Model for Shield Tunneling Advance Rate Prediction in Mixed Ground Condition Considering Past Operations

2020

View full text Add to dashboard Cite

The advance rate (AR) is a significant parameter in shield tunneling construction, which has a major impact on construction efficiency. From a practical perspective, it's helpful to establish a predictive model of the AR, which takes into account the instantaneous parameters as well as the past operations. However, for shield tunneling in mixed ground conditions, most researches focused on the average values of AR per ring and neglect the influence of past operations. This paper presents a long short-term memory (LSTM) recurrent neural network model, which was developed for the slurry shield tunneling in a mixed ground of round gravel and mudstone in Nanning metro. A temporal aggregated random forest is employed to rank the importance of the explanatory features. The model performances in different ground conditions are investigated. The results show that the LSTM model can be effectively implemented for the AR prediction. A high correlation is observed between predicted and measured AR with a correlation coefficient (R 2) of 0.93. The LSTM based AR predictive model is compared with the random forest (RF) model, the deep feedforward network (DFN) model, and the support vector regression (SVR) model. The comparison shows that the LSTM model has the best performances compared to other models. With onefourth features, we can achieve a 95% prediction accuracy measured by the R 2 in the proposed model.

show abstract

“…(Esri, 2020). Surface water and watershed boundaries obtained from © Scholars GeoPortal (DMTI Spatial Inc., 2014a, b, c, 2019 and the © TRCA (Toronto and Region Conservation Authority, 2020b) Finally, there is considerable evidence that ensemble-based and resampling techniques to improve prediction accuracy on infrequent samples such as high flows (Galar et al, 2012). Ensemble methods, such as bootstrap aggregating (Bagging) and 60 boosting, are known for their ability to improve model generalisation.…”

Section: Introductionmentioning

confidence: 99%

Resampling and ensemble techniques for improving ANN-based high streamflow forecast accuracy

Snieder¹,

Abogadil²,

Khan³

2020

Preprint

View full text Add to dashboard Cite

Abstract. Data-driven flow forecasting models, such as Artificial Neural Networks (ANNs), are increasingly used for operational flood warning systems. However, flow distributions are highly imbalanced, resulting in poor prediction accuracy on high flows, both in terms of amplitude and timing error. Resampling and ensemble techniques have shown to improve model performance of imbalanced datasets such as streamflow. In this research, we systematically evaluate and compare three resampling: random undersampling (RUS), random oversampling (ROS), and SMOTER; and four ensemble techniques: randomised weights and biases, bagging, adaptive boosting (AdaBoost), least squares boosting (LSBoost); on their ability to improve high flow prediction accuracy using ANNs. The methods are implemented both independently and in combined, hybrid techniques. While some of these combinations have been explored in the broader machine learning literature, this research contains many of the first instances of these algorithms to address the imbalance problem inherent in flood and high flow forecasting models. Specifically, the implementation of ROS, and new approaches for SMOTER, LSBOOST, and SMOTER-AdaBoost are presented in this research. Data from two Canadian watersheds (the Bow River in Alberta, and the Don River in Ontario), representing distinct hydrological systems, are used as the basis for the comparison of the methods. The models are evaluated on overall performance and on high flows. The results of this research indicate that resampling produces marginal improvements to high flow prediction accuracy, whereas ensemble methods produce more substantial improvements, with or without a resampling method. Compared to simple ANN flow forecast models, the use of ensemble methods is recommended to reduce the amplitude and timing error in highly imbalanced flow datasets.

show abstract

Comparison of learning-based wastewater flow prediction methodologies for smart sewer management

Cited by 28 publications

References 19 publications

Resampling and ensemble techniques for improving ANN-based high-flow forecast accuracy

Resampling and ensemble techniques for improving ANN-based high-flow forecast accuracy

Deep Learning Model for Shield Tunneling Advance Rate Prediction in Mixed Ground Condition Considering Past Operations

Resampling and ensemble techniques for improving ANN-based high streamflow forecast accuracy

Contact Info

Product

Resources

About