A Multi-Attention UNet for Semantic Segmentation in Remote Sensing Images

Abstract

In recent years, with the development of deep learning, semantic segmentation for remote sensing images has gradually become a hot issue in computer vision. However, segmentation for multicategory targets is still a difficult problem. To address the issues regarding poor precision and multiple scales in different categories, we propose a UNet, based on multi-attention (MA-UNet). Specifically, we propose a residual encoder, based on a simple attention module, to improve the extraction capability of the backbone for fine-grained features. By using multi-head self-attention for the lowest level feature, the semantic representation of the given feature map is reconstructed, further implementing fine-grained segmentation for different categories of pixels. Then, to address the problem of multiple scales in different categories, we increase the number of down-sampling to subdivide the feature sizes of the target at different scales, and use channel attention and spatial attention in different feature fusion stages, to better fuse the feature information of the target at different scales. We conducted experiments on the WHDLD datasets and DLRSD datasets. The results show that, with multiple visual attention feature enhancements, our method achieves 63.94% mean intersection over union (IOU) on the WHDLD datasets; this result is 4.27% higher than that of UNet, and on the DLRSD datasets, the mean IOU of our methods improves UNet’s 56.17% to 61.90%, while exceeding those of other advanced methods. The implementation code is available on the following Github Link.

Zero-Shot Image Classification Method Based on Attention Mechanism and Semantic Information Fusion

“…Sun et al and Liu et al applied an attention mechanism in the semantic segmentation task of remote sensing images. They proposed a multi-attention-based UNet [ 41 ] and an attention-based residual encoder [ 42 ], respectively. Through channel attention and spatial attention, the capability of fine-grained features was improved.…”

Anti-Aliasing Attention U-net Model for Skin Lesion Segmentation

“…ResUnet++ is an extension of the u-net architecture that incorporates residual connections in the encoder and the decoder paths [ 29 ]. DoubleU-Net is another variant of the u-net architecture that includes Atrous Spatial Pyramid Pooling (ASPP) to selectively highlight contextual features [ 30 ]. Deep Residual u-net is a variant of the u-net architecture that contains residual connections and deep supervision to improve segmentation accuracy [ 31 ].…”

ResAt-UNet: A U-Shaped Network Using ResNet and Attention Module for Image Segmentation of Urban Buildings

“…The past decade has witnessed the continuous development of neural network, which is also used in cloud segmentation [8-[13], power transmission system [14], and building segmentation [15][16][17][18][19][20], etc. In 2015, Long et al [21] proposed the Fully Convolutional Network (FCN), which was the first network successfully used for image segmentation.…”

“…Zhao et al [24] proposed the Pyramid Scene Parsing Network (PSPNet), which used a pyramid pooling module to extract contextual information and stacked the contextual information with the extracted feature information to complete image segmentation, urban buildings are obscured by shadows on high-resolution remote sensing images, and the network 's extraction of object information is incomplete. Sun et al [17] proposed a Multi-Attention UNet (MA-UNet) based on the UNet network, adding a residual encoder with an attention mechanism and a self-attention mechanism. However, due to the complex background of the ground features and irregular boundaries, this method will produce some misclassifications and omissions during extraction.…”

“…CombinedLoss= λ* BceLoss+ μ* DiceLoss( λ+μ=1) (17) In the equation, 𝑥 1 indicates the predicted target category and 𝑦 1 indicates the actual target category.…”