UATNet: U-Shape Attention-Based Transformer Net for Meteorological Satellite Cloud Recognition

Abstract

Cloud recognition is a basic task in ground meteorological observation. It is of great significance to accurately identify cloud types from long-time-series satellite cloud images for improving the reliability and accuracy of weather forecasting. However, different from ground-based cloud images with a small observation range and easy operation, satellite cloud images have a wider cloud coverage area and contain more surface features. Hence, it is difficult to effectively extract the structural shape, area size, contour shape, hue, shadow and texture of clouds through traditional deep learning methods. In order to analyze the regional cloud type characteristics effectively, we construct a China region meteorological satellite cloud image dataset named CRMSCD, which consists of nine cloud types and the clear sky (cloudless). In this paper, we propose a novel neural network model, UATNet, which can realize the pixel-level classification of meteorological satellite cloud images. Our model efficiently integrates the spatial and multi-channel information of clouds. Specifically, several transformer blocks with modified self-attention computation (swin transformer blocks) and patch merging operations are used to build a hierarchical transformer, and spatial displacement is introduced to construct long-distance cross-window connections. In addition, we introduce a Channel Cross fusion with Transformer (CCT) to guide the multi-scale channel fusion, and design an Attention-based Squeeze and Excitation (ASE) to effectively connect the fused multi-scale channel information to the decoder features. The experimental results demonstrate that the proposed model achieved 82.33% PA, 67.79% MPA, 54.51% MIoU and 70.96% FWIoU on CRMSCD. Compared with the existing models, our method produces more precise segmentation performance, which demonstrates its superiority on meteorological satellite cloud recognition tasks.

A brief analysis of remote sensing image cloud extraction method based on deep learning

“…In order to improve the accuracy of thin cloud detection, the spatial prior self-attention block (SPSA) mechanism is introduced to reconstruct the spatial location information from the deep network in the decoding stage [75] . In 2021, Wang et al [76] . proposed the model UATNet, which includes the swin transformer and channel transformer structures calculated using sliding windows.…”

“…The encoding and decoding module extracts the global context information of the image, judges the category probability of each picture element according to the fused features, and inputs it into the classifier for pixel level cloud and non cloud segmentation [27] . Later, a large number of algorithms combining U-Net and attention mechanism appeared [24,72,76] . In addition, the novelty of the image segmentation algorithm SegNet is the way in which the decoder upsamples its lower resolution input feature maps on top of the codec architecture used.…”

A Transformer-Based Coarse-to-Fine Wide-Swath SAR Image Registration Method under Weak Texture Conditions

“…Guo et al [31] applied an Attention Pyramid Network for aircraft detection. Transformer is also used in recognition [34], detection [35] and segmentation [36]. LoFTR (Local Feature TRansformer) [37] has been proposed as a coarse-to-fine image matching method based on Transformers.…”

Evaluation Model of Product Shape Design Scheme Based on Fuzzy Genetic Algorithm Mining Spatial Association Rules

“…Some scholars proposed an automatic screening method based on clustering and expert experience, realizing the practical application of association rule data mining in traffic data [6]. e application of data mining combined with Bayesian algorithm in meteorological data also has good prediction effect [7]. In the automatic mining of fuzzy sets and fuzzy association rules, the automatic clustering method based on GA has remarkable efficiency circle.…”