Swin-Conv-Dspp and Global Local Transformer for Remote Sensing Image Semantic Segmentation

Abstract

Compared with the traditional method based on hand-crafted features, deep neural network has achieved a certain degree of success on remote sensing (RS) image semantic segmentation. However, there are still serious holes, rough edge segmentation, and false detection or even missed detection due to the light and its shadow in the segmentation. Aiming at the above problems, this article proposes a RS semantic segmentation model SCG-TransNet that is a hybrid model of Swin transformer and Deeplabv3+, which includes Swin-Conv-Dspp (SCD) and global local transformer block (GLTB). First, the SCD module which can efficiently extract feature information from objects at different scales is used to mitigate the hole phenomenon, reducing the loss of detailed information. Second, we construct a GLTB with spatial pyramid pooling shuffle module to extract critical detail information from the limited visible pixels of the occluded objects, which alleviates the problem of difficult object recognition due to occlusion effectively. Finally, the experimental results show that our SCG-TransNet achieves a mean intersection over union of 70.29% on the Vaihingen datasets, which is 3% higher than the baseline model. It also achieved good results on POSDAM datasets. These demonstrate the effectiveness, robustness, and superiority of our proposed method compared with existing state-of-the-art methods.

A Bio-Inspired Visual Perception Transformer for Cross-Domain Semantic Segmentation of High-Resolution Remote Sensing Images

“…Some scholars have focused on extracting deep semantic features of high-resolution remote sensing images by capturing long-range contextual information. Mo proposed a transformer framework with a spatial pyramid pool shuffling module that can extract key details and information from limited visible pixels of occluded objects by learning long-range dependencies [17]. Peng used multi-scale context patches to guide local image patches to focus on different fine-grained objects to extract contextual features on a large scale [18].…”