Axial Cross Attention Meets CNN: Bibranch Fusion Network for Change Detection

Abstract

In the previous years, Vision Transformer has demonstrated a global information extraction capability in the field of Computer Vision that CNN lacks. Due to the lack of inductive bias in Vision Transformer, it requires a large amount of data to support its training. In the field of remote sensing, it costs a lot to obtain a significant number of high-resolution remote sensing images. Most existing change detection networks based on deep learning rely heavily on CNN, which cannot effectively utilize the long-distance dependency between pixels for difference discrimination. Therefore, this work aims to use highperformance Vision Transformer to conduct change detection research with limited data. A Bi-branch Fusion Network Based on Axial Cross Attention (ACABFNet) is proposed. The network extracts local and global information of images through the CNN Branch and Transformer Branch respectively, and then fuses local and global features by bidirectional fusion approach. In the upsampling stage, similar feature information and difference feature information of the two branches are explicitly generated by feature addition and feature subtraction. Considering that the self-attention mechanism is not efficient enough for global attention over small datasets, we propose the Axial Cross Attention. First, global attention along the height and width dimensions of images is performed respectively, and then Cross Attention is used to fuse the global feature information along two dimensions. Compared with the original self-attention, the structure is more GPU-friendly and efficient. Experiment results on three datasets reveal that ACABFNet outperforms existing change detection algorithms.

Coastal Aquaculture Area Extraction Based on Self-Attention Mechanism and Auxiliary Loss

“…The SPAM structure in SPANet is used to extract deeper multiscale features and salient features, and the FFM structure is used to complete the fusion of low-order features and high-order features. In the field of deep learning for remote sensing change monitoring, Song et al [27] proposed ACABFNet. ACABFNet extracts the local information of the image by CNN branch and the global information of the image by transformer branch, and then fuses the two by using bidirectional fusion.…”

Remote-Sensing Image Change Detection Based on Adjacent-Level Feature Fusion and Dense Skip Connections

“…Hybrid-TransCD [8] introduces the hybrid transformer structure to capture multi-granularity global context dependencies, which aggregates fine-grained change details with coarse-grained change region information and can effectively retain spectral features in complex scenes, with an F1 score of 90.06 on the LEVIR-CD dataset [9]. ACABFNet [10] extracts the local and global information of the images by CNN branch and transformer branch, respectively, and then fuses the local and global features by bidirectional fusion, with an F1 score of 90.68 on the LEVIR-CD dataset [9]. SARAS-Net [11] designed three modules: relation-awareness, scale-awareness, and cross-transformer, which effectively solves the problem of CD between different scenes, and its F1 score on the LEVIR-CD dataset reaches 91.91.…”

MCEENet: Multi-Scale Context Enhancement and Edge-Assisted Network for Few-Shot Semantic Segmentation

“…As a fundamental problem in the field of computer vision, semantic segmentation has obtained tremendous improvements during the past few years. As shown in Figure 1 , it has been widely used in medical image recognition [ 1 ], 3D points Clouds [ 2 ], geological exploration [ 3 ], cloud and cloud shadow segmentation [ 4 , 5 ], remote sensing image [ 6 , 7 , 8 , 9 ], and automatic driving [ 10 ], etc. Existing semantic segmentation models based on convolutional neural networks (CNNs) (e.g., U-Net [ 11 ] and DeepLab [ 12 ]) often rely on a large amount of pixel-level labeled data, which leads to the following two problems: (1) it costs plenty of labor due to the fact that all training samples should be marked manually one by one, and (2) these models perform poorly in recognizing novel objects.…”