Deep transfer learning based on transformer for flood forecasting in data-sparse basins

Xu, Yan; Lin, Kairong; Hu, Caihong; Wang, Shuli; Wu, Qiang; Zhang, Li; Ran, Guang

doi:10.1016/j.jhydrol.2023.129956

Cited by 38 publications

(9 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In various domains, there is work leveraging deep learning for the purpose of assimilating data from monitoring points to forecast critical parameters. Xu et al [15] presented, in the form of the transfer learning framework based on transformer (TL-Transformer), an accurate prediction of flooding in data-sparse basins. El-Shafeiy et al [16] introduced and applied a pioneering technology, multivariate multiple convolutional networks with long short-term memory (MCN-LSTM), to real-time water quality monitoring.…”

Section: Related Workmentioning

confidence: 99%

Enhancing LOCA Breach Size Diagnosis with Fundamental Deep Learning Models and Optimized Dataset Construction

Xiao,

Qi,

Liang

et al. 2023

Energies

View full text Add to dashboard Cite

In nuclear power plants, the loss-of-coolant accident (LOCA) stands out as the most prevalent and consequential incident. Accurate breach size diagnosis is crucial for the mitigation of LOCAs, and identifying the cause of an accident can prevent catastrophic consequences. Traditional methods mostly focus on combining model algorithms and utilize intricate composite model neural network architectures. However, it is crucial to investigate whether greater complexity necessarily leads to better performance. In addition, the consideration of the impact of dataset construction and data preprocessing on model performance is also needed for model building. This paper proposes a framework named DeepLOCA-Lattice to experiment with different preprocessing approaches to fundamental deep learning models for a comprehensive analysis of the diagnosis of LOCA breach size. The DeepLOCA-Lattice involves data preprocessing via the lattice algorithm and equal-interval partitioning and deep-learning-based models, including the multi-layer perceptron (MLP), recurrent neural networks (RNNs), convolutional neural networks (CNNs), and the transformer model in LOCA breach size diagnosis. After conducting rigorous ablation experiments, we have discovered that even rudimentary foundational models can achieve accuracy rates that exceed 90%. This is a significant improvement when compared to the previous models, which yield an accuracy rate of lower than 50%. The results interestingly demonstrate the superior performance and efficacy of the fundamental deep learning model, with an effective dataset construction approach. It elucidates the presence of a complex interplay among diagnostic scales, sliding window size, and sliding stride. Furthermore, our investigation reveals that the model attains its highest accuracy within the discussed range when utilizing a smaller sliding stride size and a longer sliding window length. This study could furnish valuable insights for constructing models for LOCA breach size estimation.

show abstract

Section: Related Workmentioning

confidence: 99%

Enhancing LOCA Breach Size Diagnosis with Fundamental Deep Learning Models and Optimized Dataset Construction

Xiao,

Qi,

Liang

et al. 2023

Energies

View full text Add to dashboard Cite

show abstract

“…However, in the context of ood forecasting, there are only a handful of studies to be found that involve Transformer architecture. Further, some of these studies show that accuracy comparison of models including Transformers differs for different datasets (Wei et al, 2023;Xu et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

Multi Day Ahead Flood Predictionin South Asian Tropical Zone Using Deep Learning

Madhushanka,

Jayasinghe,

Rajapakse

2024

Preprint

View full text Add to dashboard Cite

A reliable and accurate flood forecasting procedure is a critical need due to the hazardous nature of the disaster. Researchers are increasingly favoring innovative approaches with enhanced accuracy, such as machine learning models, over traditional methods for this task. However, lack of such studies regarding South Asian tropical region, which has its own climate characteristics, was unidentified as a major issue. This research delves into the viability of employing ANN, LSTM, BLSTM, ConvLSTM2D and Transformer models for multi-day ahead flood simulation. One-day, two-days and three-days were selected as lead times for the task considering the lower reaches of the Mahaweli catchment in Sri Lanka, which is mostly affected by the Northeast Monsoon. The prediction capability of extreme stream flows was also of interest. Observed rainfall data from three nearby rain gauges, along with historical discharges of the target river gauge, serve as input features for the models. The ANN model showed the worst performance, having the mean NSE of 0.67. An improved performance was observed from the Transformer compared to the LSTM based models, especially in multiple day ahead forecasting scenarios. For all the models, the forecasting capability of extreme water levels drops down drastically when the prediction lead time is increased.

show abstract

“…In the realm of ood forecasting, there is a scarcity of studies that incorporate Transformer architecture, representing a notable gap in the literature. Moreover, existing research indicates that the accuracy comparison of models, including Transformers, varies across different datasets (Wei et al, 2023;Xu et al, 2023).…”

Section: Introductionmentioning

confidence: 99%

“…LSTM and Transformer encoder shows similar accuracies in daily water level forecasting and predicting extremities. Although Transformer models tend to show a better performance in stream ow forecasting as in studies of(Castangia et al, 2023;Liu et al, 2022;Xu et al, 2023), lack of su cient data may cause the performance reduction for the Transformer Encoder model(Wei et al, 2023).In this case, switching the inputs between the Encoder and Decoder shows similar performance but it might be different in another case. Therefore, it is recommended to do this strategy and check the accuracy when there are input features with 2 different time steps.DeclarationsAuthor Contribution G.W.T.I.…”

mentioning

confidence: 99%

Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate

Madhushanka,

Jayasinghe,

Rajapakse

2024

Preprint

View full text Add to dashboard Cite

The imperative for a reliable and accurate flood forecasting procedure stem from the hazardous nature of the disaster. In response, researchers are increasingly turning to innovative approaches, particularly machine learning models, which offer enhanced accuracy compared to traditional methods. However, a notable gap exists in the literature concerning studies focused on the South Asian tropical region, which possesses distinct climate characteristics. This study investigates the applicability and behavior of Long Short-Term Memory (LSTM) and Transformer models in flood simulation with one day lead time, at the lower reach of Mahaweli catchment in Sri Lanka, which is mostly affected by the Northeast Monsoon. The importance of different input variables in the prediction was also a key focus of this study. Input features for the models included observed rainfall data collected from three nearby rain gauges, as well as historical discharge data from the target river gauge. Results showed that use of past water level data denotes a higher impact on the output compared to the other input features such as rainfall, for both architectures. All models denoted satisfactory performances in simulating daily water levels, especially low stream flows, with Nash Sutcliffe Efficiency (NSE) values greater than 0.77 while Transformer Encoder model showed a superior performance compared to Encoder Decoder models.

show abstract

Deep transfer learning based on transformer for flood forecasting in data-sparse basins

Cited by 38 publications

References 61 publications

Enhancing LOCA Breach Size Diagnosis with Fundamental Deep Learning Models and Optimized Dataset Construction

Enhancing LOCA Breach Size Diagnosis with Fundamental Deep Learning Models and Optimized Dataset Construction

Multi Day Ahead Flood Predictionin South Asian Tropical Zone Using Deep Learning

Behavior of LSTM and Transformer Deep Learning Models in Flood Simulation Considering South Asian Tropical Climate

Contact Info

Product

Resources

About