Controllably Deep Supervision and Multi-Scale Feature Fusion Network for Cloud and Snow Detection Based on Medium- and High-Resolution Imagery Dataset

Abstract

Clouds and snow in remote sensing imageries cover underlying surface information, reducing image availability. Moreover, they interact with each other, decreasing the cloud and snow detection accuracy. In this study, we propose a convolutional neural network for cloud and snow detection, named the cloud and snow detection network (CSD-Net). It incorporates the multi-scale feature fusion module (MFF) and the controllably deep supervision and feature fusion structure (CDSFF). MFF can capture and aggregate features at various scales, ensuring that the extracted high-level semantic features of clouds and snow are more distinctive. CDSFF can provide a deeply supervised mechanism with hinge loss and combine information from adjacent layers to gain more representative features. It ensures the gradient flow is more oriented and error-less, while retaining more effective information. Additionally, a high-resolution cloud and snow dataset based on WorldView2 (CSWV) was created and released. This dataset meets the training requirements of deep learning methods for clouds and snow in high-resolution remote sensing images. Based on the datasets with varied resolutions, CSD-Net is compared to eight state-of-the-art deep learning methods. The experiment results indicate that CSD-Net has an excellent detection accuracy and efficiency. Specifically, the mean intersection over the union (MIoU) of CSD-Net is the highest in the corresponding experiment. Furthermore, the number of parameters in our proposed network is just 7.61 million, which is the lowest of the tested methods. It only has 88.06 GFLOPs of floating point operations, which is less than the U-Net, DeepLabV3+, PSPNet, SegNet-Modified, MSCFF, and GeoInfoNet. Meanwhile, CSWV has a higher annotation quality since the same method can obtain a greater accuracy on it.

UCTNet with Dual-Flow Architecture: Snow Coverage Mapping with Sentinel-2 Satellite Imagery

“…First, we illustrate the performance of the proposed UCTNet on the previously stated snow/cloud dataset. As shown in Table 2, a variety of methods are used for the performance comparison, including commonly used segmentation methods and the specially designed snow and cloud detection method (i.e., CSD-Net [35]). Segmentation methods could be categorized into CNN-based methods (i.e., U-Net [36] and DeepLab-V3 [37]) and recently proposed Transformer-based networks (i.e., ResT-Tiny [38] and Swin-Tiny [27]).…”

Multilevel feature context semantic fusion network for cloud and cloud shadow segmentation

“…Table 4 shows the values of various indicators of different models on our dataset. In order to fully prove the advantages of our model in identifying clouds and cloud shadows, in addition to the classical semantic segmentation model, we select different models in the past five years as a comparison, among which we also select four latest networks for cloud detection: SP_CSANet, 43 CSDNet, 45 PADANet, 41 and GAFRNet 42 . It can be seen from this table that whether it is the semantic segmentation network or the network model dedicated to cloud detection, the proposed model has the highest accuracy in all indicators, at least 1.949% higher than the other networks on MIOU.…”

A brief analysis of remote sensing image cloud extraction method based on deep learning

“…MFF can obtain low-level features with different fine-grained and high-level features with different sizes to ensure that the highlevel semantic features of cloud and snow extracted are more distinct. CDSFF can provide a depth-supervised mechanism with adjustable losses and combine information from neighbouring layers to obtain more representative features [43] .…”