Transformer-based multi-scale feature fusion network for remote sensing change detection

Abstract

Change detection (CD) is the operation of quantitatively analyzing the surface changes of a phenomenon or objects over two different times. Lately, CD based on deep learning has developed to become more and more powerful, and convolutional neural networks (CNNs) have dominated the field of remote sensing (RS) CD. In particular, in many fields of computer vision, neural networks based on U-Net network and skip connections have been generally used. However, despite the excellent performance achieved by CNN, it does not learn global and longrange semantic information interaction well due to the locality of convolutional operations. The recently proposed Swin-UNet in the field of medical image segmentation achieved excellent results, which is a U-Net-like pure transformer. In the face of the challenge of segmentation accuracy, the Swin transformer has demonstrated strong capabilities. The Swin transformer block (STB) consists of residual connected STBs used in SwinIR to enhanced training stability. We began to try to incorporate them into our network for RS CD. Finally, we propose a transformer-based multi-scale feature fusion model (TMFF), including decoder, encoder, and skip connection structure, for RS image CD. We modify the original U-Net architecture so that it can better aggregate semantic features at all levels. Our proposed TMFF achieves impressive results through experiments on three datasets; © 2022 Society of Photo-Optical Instrumentation Engineers (SPIE)

Guided filter-based self-supervised remote sensing image change detection with global contrast

“…Among these deep learning methods, supervised learning methods have performed successfully in remote sensing image CD due to their powerful modeling and learning capabilities 12 – 16 For example, Deng et al 15 . incorporated depthwise separable convolution and multi-headed self-attention in UNet++, which effectively enhances the network’s ability to extract local and global information.…”

“…incorporated depthwise separable convolution and multi-headed self-attention in UNet++, which effectively enhances the network’s ability to extract local and global information. Yan et al 16 . proposed a multilevel feature aggregation and enhancement network to sense information at different scales and capture their dependencies.…”

DenseTrans: Multimodal Brain Tumor Segmentation Using Swin Transformer

“…However, because the current segmentation methods based on brain tumors mostly rely on CNN and its variants, although CNN has achieved excellent performance, it cannot learn global and remote semantic information interaction well due to the locality of convolutional operation [13], [14], [15], lacks the ability to model long-term dependencies explicitly. later, although some researchers have introduced the brilliant transformer architecture in the field of Natural Language Processing(NLP) into the field of image segmentation, transformer and its variants, such as vision transformer, require a large number of data sets for pre-training.…”