Landslide Image Captioning Method Based on Semantic Gate and Bi-Temporal LSTM

Cui, Wenqi; He, Xin; Meng, Yan; Wang, Ziwei; Li, Jie; Hao, Yuanjie; Wu, Weijie; Zhao, Huiling; Chen, Xianfeng; Cui, Wei

doi:10.3390/ijgi9040194

Cited by 15 publications

(16 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, Bidirectional-LSTM was employed to learn the factor data of forward and backward information. This Bi-LSTM aimed to model the inter-relationship among the factors from sequential observed data [ 44 , 45 ].…”

Section: Methodsmentioning

confidence: 99%

Automated Landslide-Risk Prediction Using Web GIS and Machine Learning Models

Tengtrairat

Woo

Parathai

et al. 2021

Sensors

View full text Add to dashboard Cite

Spatial susceptible landslide prediction is the one of the most challenging research areas which essentially concerns the safety of inhabitants. The novel geographic information web (GIW) application is proposed for dynamically predicting landslide risk in Chiang Rai, Thailand. The automated GIW system is coordinated between machine learning technologies, web technologies, and application programming interfaces (APIs). The new bidirectional long short-term memory (Bi-LSTM) algorithm is presented to forecast landslides. The proposed algorithm consists of 3 major steps, the first of which is the construction of a landslide dataset by using Quantum GIS (QGIS). The second step is to generate the landslide-risk model based on machine learning approaches. Finally, the automated landslide-risk visualization illustrates the likelihood of landslide via Google Maps on the website. Four static factors are considered for landslide-risk prediction, namely, land cover, soil properties, elevation and slope, and a single dynamic factor i.e., precipitation. Data are collected to construct a geospatial landslide database which comprises three historical landslide locations—Phu Chifa at Thoeng District, Ban Pha Duea at Mae Salong Nai, and Mai Salong Nok in Mae Fa Luang District, Chiang Rai, Thailand. Data collection is achieved using QGIS software to interpolate contour, elevation, slope degree and land cover from the Google satellite images, aerial and site survey photographs while the physiographic and rock type are on-site surveyed by experts. The state-of-the-art machine learning models have been trained i.e., linear regression (LR), artificial neural network (ANN), LSTM, and Bi-LSTM. Ablation studies have been conducted to determine the optimal parameters setting for each model. An enhancement method based on two-stage classifications has been presented to improve the landslide prediction of LSTM and Bi-LSTM models. The landslide-risk prediction performances of these models are subsequently evaluated using real-time dataset and it is shown that Bi-LSTM with Random Forest (Bi-LSTM-RF) yields the best prediction performance. Bi-LSTM-RF model has improved the landslide-risk predicting performance over LR, ANNs, LSTM, and Bi-LSTM in terms of the area under the receiver characteristic operator (AUC) scores by 0.42, 0.27, 0.46, and 0.47, respectively. Finally, an automated web GIS has been developed and it consists of software components including the trained models, rainfall API, Google API, and geodatabase. All components have been interfaced together via JavaScript and Node.js tool.

show abstract

Section: Methodsmentioning

confidence: 99%

Automated Landslide-Risk Prediction Using Web GIS and Machine Learning Models

Tengtrairat

Woo

Parathai

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Furthermore, J. Aneja [21] and S. J. Rennie [25] adapted the attention mechanism to generate caption. For vision part of image captioning Vgg16 were used by most of the papers [2], [11], [24], [25], [27], [30] as CNN but some of them also used YOLO [9], Inception V3 [6], [31], AlexNet [24], [30] ResNet [11], [18], [24] or Unet [4] as CNN for feature extraction. Concurrently, LSTM [6], [9], [11], [17], [31] was used by most of the papers for generating the next word in the sequence.…”

Section: Related Workmentioning

confidence: 99%

“…Concurrently, LSTM [6], [9], [11], [17], [31] was used by most of the papers for generating the next word in the sequence. However, some of the researcher also utilized RNN [19] or BiLSTM [4], [30]. Moreover, P. Blandfort et al [32] systematically characterize diverse image captions that appear "in the wild" in order to understand how people caption images naturally.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

A Hybridized Deep Learning Method for Bengali Image Captioning

Humaira¹,

Paul²,

Abidur³

et al. 2021

IJACSA

View full text Add to dashboard Cite

An omnipresent challenging research topic in computer vision is the generation of captions from an input image. Previously, numerous experiments have been conducted on image captioning in English but the generation of the caption from the image in Bengali is still sparse and in need of more refining. Only a few papers till now have worked on image captioning in Bengali. Hence, we proffer a standard strategy for Bengali image caption generation on two different sizes of the Flickr8k dataset and BanglaLekha dataset which is the only publicly available Bengali dataset for image captioning. Afterward, the Bengali captions of our model were compared with Bengali captions generated by other researchers using different architectures. Additionally, we employed a hybrid approach based on InceptionResnetV2 or Xception as Convolution Neural Network and Bidirectional Long Short-Term Memory or Bidirectional Gated Recurrent Unit on two Bengali datasets. Furthermore, a different combination of word embedding was also adapted. Lastly, the performance was evaluated using Bilingual Evaluation Understudy and proved that the proposed model indeed performed better for the Bengali dataset consisting of 4000 images and the BanglaLekha dataset.

show abstract

“…Similarly, a stacked LSTM model was developed by stacking LSTM layers to forecast the soil movements (Xing et al, 2019). Beyond some of these attempts for forecasting soil movements, there have been some attempts at developing RNN models for the time series forecasting problems across different domains (Huang et al, 2015;Behera et al, 2018Behera et al, , 2021bQiu et al, 2018;Zhang et al, 2019;Barzegar et al, 2020;Cui et al, 2020;Singh et al, 2020). For example, convolutional LSTM (Conv-LSTM), bidirectional LSTM (Bi-LSTM), and CNN-LSTM models have been developed in the natural language processing (NLP), crowd time series forecasting, software reliability assessment, and water quality variable forecasting (Huang et al, 2015;Behera et al, 2018Behera et al, , 2021bQiu et al, 2018;Zhang et al, 2019;Barzegar et al, 2020;Cui et al, 2020;Singh et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

BS-LSTM: An Ensemble Recurrent Approach to Forecasting Soil Movements in the Real World

et al. 2021

View full text Add to dashboard Cite

Machine learning (ML) proposes an extensive range of techniques, which could be applied to forecasting soil movements using historical soil movements and other variables. For example, researchers have proposed recurrent ML techniques like the long short-term memory (LSTM) models for forecasting time series variables. However, the application of novel LSTM models for forecasting time series involving soil movements is yet to be fully explored. The primary objective of this research is to develop and test a new ensemble LSTM technique (called “Bidirectional-Stacked-LSTM” or “BS-LSTM”). In the BS-LSTM model, forecasts of soil movements are derived from a bidirectional LSTM for a period. These forecasts are then fed into a stacked LSTM to derive the next period’s forecast. For developing the BS-LSTM model, datasets from two real-world landslide sites in India were used: Tangni (Chamoli district) and Kumarhatti (Solan district). The initial 80% of soil movements in both datasets were used for model training and the last 20% of soil movements in both datasets were used for model testing. The BS-LSTM model’s performance was compared to other LSTM variants, including a simple LSTM, a bidirectional LSTM, a stacked LSTM, a CNN-LSTM, and a Conv-LSTM, on both datasets. Results showed that the BS-LSTM model outperformed all other LSTM model variants during training and test in both the Tangni and Kumarhatti datasets. This research highlights the utility of developing recurrent ensemble models for forecasting soil movements ahead of time.

show abstract

Landslide Image Captioning Method Based on Semantic Gate and Bi-Temporal LSTM

Cited by 15 publications

References 69 publications

Automated Landslide-Risk Prediction Using Web GIS and Machine Learning Models

Automated Landslide-Risk Prediction Using Web GIS and Machine Learning Models

A Hybridized Deep Learning Method for Bengali Image Captioning

BS-LSTM: An Ensemble Recurrent Approach to Forecasting Soil Movements in the Real World

Contact Info

Product

Resources

About