Improved Swin Transformer-Based Semantic Segmentation of Postearthquake Dense Buildings in Urban Areas Using Remote Sensing Images

Abstract

Timely acquiring accurate information on the earthquake-induced damage of buildings is crucial for emergency assessment and post-disaster rescue. Optical remote sensing photography has been a typical method for obtaining seismic data in the early stage after an earthquake due to its wide coverage and fast response speed. Currently, convolutional neural networks (CNNs) are widely applied for remote sensing image recognition. However, insufficient extraction and expression ability of global correlations between local image patches limit the performance of dense building segmentation under real-world scenarios. To address the above issue, this paper proposes an improved Swin Transformer to segment dense urban buildings from remote sensing images with complex backgrounds. The original Swin Transformer is used as a backbone of the encoder, and a convolutional block attention module (CBAM) is employed in the linear embedding and patch merging stages to focus on significant features. Hierarchical feature maps are first fused by convolution operators to strengthen the feature extraction process and adjust the channel of feature dimension, followed by being fed to the UPerNet (as the decoder) to obtain the final segmentation map. Buildings are labeled from the collected remote sensing images of the Yushu and Beichuan earthquakes and classified into collapsed and non-collapsed buildings. Data augmentation transformations of horizontal and vertical flipping, brightness adjustment, uniform fogging, and non-uniform fogging are performed to simulate different photography directions, light overexposure, darkness, and foggy occlusions under actual situations. The effectiveness and superiority of the proposed method over the original Swin Transformer and several mature CNN-based segmentation models are validated by ablation experiments and comparative studies. The results show that the mean intersection over union (mIoU) of the improved Swin Transformer reaches 88.53%, achieving a significant improvement of 1.3% compared to the original Swin Transformer of 87.23%. Furthermore, the improved Swin Transformer can accurately identify dense buildings in remote sensing images under complex weather disturbances. Finally, the results indicate good stability, robustness, and generalization ability of the proposed method in the dense segmentation of seismic buildings.

FishSegSSL: A Semi-Supervised Semantic Segmentation Framework for Fish-Eye Images

“…Vision transformers are gaining popularity in semantic segmentation for the advantage gained from their special network architecture. These models learn pixel-level feature representation using a transformer-based encoder, an attention mechanism, and a bottle-neck layer [ 19 , 20 ]. SegViT [ 21 ] stands apart from conventional ViTs because it uses an Attention-to-Mask (ATM) module.…”

Enhancing Efficient Global Understanding Network With CSWin Transformer for Urban Scene Images Segmentation

“…Furthermore, researchers [8], [16] have also tried to use Swin Transformer as a backbone network in combination with CNNs for RS imaging segmentation tasks. In this regard, related research works [60], [61], [62], [63] have indicated that Swin Transformer as an encoder can be combined with different decoder architectures (e.g., Unet [13], PSP [20], and FPN [64]) for diverse tasks to achieve the optimal segmentation outcomes. Besides, UnetFormer [65], the current state-of-the-art (SoTA) network for semantic segmentation in RS, proposes a hybrid Transformer and CNN lightweight network for real-time urban scene segmentation, which looks similar to ours but is quite different.…”

Coarse-to-Fine Seismic Assessment Based on Computer Vision

“…The multi-scale features of densely distributed buildings are extracted and fused, and three statistics (e.g., the angular second moment, dissimilarity, and inverse difference moment) are further discovered based on the gray-level cooccurrence matrix as the texture features to distinguish damage intensities of buildings. Furthermore, a novel improved Swin Transformer is proposed to segment dense urban buildings at pixel level by remote sensing images with complex backgrounds, as shown in Figure 3 [13]. The original Swin Transformer is utilized as the backbone of the encoder, and a convolutional block attention module is inserted at patch embedding and patch merging stages.…”