Real-time probabilistic forecasting of river water quality under data missing situation: Deep learning plus post-processing techniques

Abstract

Quantifying the uncertainty of probabilistic water quality forecasting induced by missing input data is fundamentally challenging. This study introduced a novel methodology for probabilistic water quality forecasting conditional on point forecasts.A Multivariate Bayesian Uncertainty Processor (MBUP) was adopted to probabilistically model the relationship between the point forecasts made by a deep learning artificial neural network (ANN) and their corresponding observed water quality. The methodology was tested using hourly water quality series at an island of Shanghai City in China. The novelties relied upon: firstly, the use of a transfer learning algorithm to overcome flatten-and under-prediction bottlenecks of river water quality raised in artificial neural networks, and secondly, the use of the MBUP to capture the dependence structure between observations and forecasts. Two deep learning ANNs were used to make the point forecasts. Then the MBUP approach driven by the point forecasts demonstrated its competency in improving the accuracy of probabilistic water quality forecasts significantly, where predictive distributions encountered in multi-step-ahead water quality forecasts were effectively reduced to small ranges. The results demonstrated that the deep learning plus the post-processing approach suitably extracted the complex dependence structure between the model's output and observed water quality so that model reliability (Containing Ratio > 85% and average Relative Band-width < 0.25) as well as forecast accuracy (Nash-Sutcliffe Efficiency coefficient > 0.8 and Root-Mean-Square-Error < 0.4 mg/l) for future horizons from 1 hour up to 10 hours were significantly improved, even if the input data missing rate reaches 50%.

Decline in net primary productivity caused by severe droughts: evidence from the Pearl River basin in China

“…The upstream is characterized by karst topography which can trigger severe droughts (Guo et al 2013). Hydrological characteristics over the karst region are found eventually altered by rocky desertification, particularly the rainfall-runoff process (Zhou 2020a). Rocky desertification features with little vegetation and soil breakage increase infiltration to subsurface systems reducing flow storage capacity and residence time and decreasing the resistance to overland flow, causing frequent drought and flood events in karst regions Yan & Cai 2015).…”

Suspended sediment load prediction using long short-term memory neural network

“…The advantage of this structure is that it speeds down the gradient vanishing. LSTM network was found to capture temporal correlations 53 .

Figure 4 LSTM cell 54 .

…”

A novel hybrid XAJ-LSTM model for multi-step-ahead flood forecasting

“…The LSTM neural network was proposed by Hochreiter & Schmidhuber (1997), and is a special recurrent neural network (RNN). The difference between LSTM and other ANNs is that the hidden layer in LSTM is composed of an internal self-loop unit (Zhou 2020), which is able to overcome the gradient explosion and disappearance bottleneck prone to appear in RNN in the backpropagation through time (BPTT) algorithm (Kao et al 2020). Therefore, LSTM has good applicability in processing the prediction of various time series.…”

“…The input variables determined need to be normalized to eliminate the influence of magnitude, thereby improving the accuracy and efficiency of network learning (Zhou 2020), and the min-max (min-max normalization) method in Equation (1):…”