A Novel Weakly Supervised Remote Sensing Landslide Semantic Segmentation Method: Combining CAM and cycleGAN Algorithms

Abstract

With the development of deep learning algorithms, more and more deep learning algorithms are being applied to remote sensing image classification, detection, and semantic segmentation. The landslide semantic segmentation of a remote sensing image based on deep learning mainly uses supervised learning, the accuracy of which depends on a large number of training data and high-quality data annotation. At this stage, high-quality data annotation often requires the investment of significant human effort. Therefore, the high cost of remote sensing landslide image data annotation greatly restricts the development of a landslide semantic segmentation algorithm. Aiming to resolve the problem of the high labeling cost of landslide semantic segmentation with a supervised learning method, we proposed a remote sensing landslide semantic segmentation with weakly supervised learning method combing class activation maps (CAMs) and cycle generative adversarial network (cycleGAN). In this method, we used the image level annotation data to replace pixel level annotation data as the training data. Firstly, the CAM method was used to determine the approximate position of the landslide area. Then, the cycleGAN method was used to generate the fake image without a landslide, and to make the difference with the real image to obtain the accurate segmentation of the landslide area. Finally, the pixel-level segmentation of the landslide area on remote sensing image was realized. We used mean intersection-over-union (mIOU) to evaluate the proposed method, and compared it with the method based on CAM, whose mIOU was 0.157, and we obtain better result with mIOU 0.237 on the same test dataset. Furthermore, we made a comparative experiment using the supervised learning method of a u-net network, and the mIOU result was 0.408. The experimental results show that it is feasible to realize landslide semantic segmentation in a remote sensing image by using weakly supervised learning. This method can greatly reduce the workload of data annotation.

1D Gradient-Weighted Class Activation Mapping, Visualizing Decision Process of Convolutional Neural Network-Based Models in Spectroscopy Analysis

“…Besides, the manual adjustment of the parameters (e.g., window size and step size) is time-costly and professional. The subsequently developed CAM could accurately visualize the inherent weights of the convolutional neural network (CNN), , which evaluates the contribution of features in the model by calculating the weights of the final fully connected layer in a CNN structure containing global average pooling (GAP) layers. The further developed Grad-CAM (including its variants) extends CAM to CNN without a GAP layer structure.…”

Lightweight Landslide Detection Network for Emergency Scenarios

“…Landslide disasters are one of the most widespread and severe natural disasters in the world. Landslide refers to the phenomenon of deformation and failure of slope rock and soil mass mainly moving horizontally along a certain surface under the action of gravity or other forces (e.g., superimposed seismic force or water pressure) [1]. Landslides pose a major threat to residential buildings, roads, service facilities, and personal safety at the foot of the mountain [1].…”

“…Landslide refers to the phenomenon of deformation and failure of slope rock and soil mass mainly moving horizontally along a certain surface under the action of gravity or other forces (e.g., superimposed seismic force or water pressure) [1]. Landslides pose a major threat to residential buildings, roads, service facilities, and personal safety at the foot of the mountain [1]. Moreover, the flow of landslides will destroy farmland and cultivated land, causing economic losses, blocking rivers to form barrier lakes, and burying hidden dangers.…”

“…Although new algorithms continue to improve the performance of landslide detection, there are still some issues that need to be resolved: (1) The feature extraction ability of traditional landslide detection networks is often improved by increasing the depth of the model. However, while improving the accuracy of detection, this will also increase the number of model parameters.…”

Semantic segmentation of remote sensing images based on dilated convolution and spatial-channel attention mechanism

“…Semantic segmentation is a critical technology in remote sensing image interpretation, as it enables pixel-by-pixel classification of objects in images. Due to its advanced classification capabilities, semantic segmentation has found wide application in many fields, such as landslide monitoring, 1 , 2 plant disease monitoring, 3 and water quality monitoring, 4 in the field of disaster monitoring. In urban planning, it has been utilized for road detection, 5 building change detection, 6 illegal building detection, 7 3D scene reconstruction 8 , 9 .…”