A CNN-Transformer Network With Multiscale Context Aggregation for Fine-Grained Cropland Change Detection

Abstract

Non-agriculturalization incidents are serious threats to local agricultural ecosystem and global food security. Remote sensing change detection (CD) can provide an effective approach for in-time detection and prevention of such incidents. However, existing CD methods are difficult to deal with the large intra-class differences of cropland changes in high-resolution images (HRIs). In addition, traditional CNN based models are plagued by the loss of long-range context information, and the high computational complexity brought by deep layers. Therefore, we propose a CNNtransformer network with multi-scale context aggregation (MSCANet), which combines the merits of CNN and transformer to fulfill efficient and effective cropland change detection. In the MSCANet, a CNN-based feature extractor is first utilized to capture hierarchical features, then a transformer-based multiscale context aggregator (MSCA) is designed to encode and aggregate context information. Finally, a multi-branch prediction head (MBPH) with three CNN classifiers is applied to obtain change maps, to enhance the supervision for deep layers. Besides, for the lack of change detection dataset with fine-grained cropland change of interest, we also provide a new cropland change detection dataset (CLCD), which contains 600 pairs of 512×512 bitemporal images with the spatial resolution of 0.5-2m. Comparative experiments with several CD models prove the effectiveness of the MSCANet, with the highest F1 of 64.67% on the high-resolution semantic change detection dataset (HRSCD), and of 71.29% on CLCD. Code and dataset in the paper will be available for download from the following link https://github.com/liumency/CropLand-CD.

Spectral Token Guidance Transformer for Multisource Images Change Detection

“…Recently, many ViT-based methods are proposed to extract robust and discriminative features to represent RS images during CD tasks. Li et al [34] proposed a CNN-Transformer network to fulfill efficient cropland CD results, where they combine the merits of CNN and transformer to fuse the multiscale context information. Zhang et al [35] designed a pure transformer network named SwinSUNet with a Siamese U-shaped structure to realize CD.…”

“…For the purpose of demonstrating the superiority of STCD-Former, we choose five representative methods for comparison: SVM, M3D-DCNN [34], ViT-spectral, ViT-spatial, and SST-Former [33]. We ditto used 0.5% train samples to train these networks.…”

Contour-enhanced densely connected Siamese network for change detection

“…In addition, transformers, 40 which were originally widely used in natural language processing, have also shown performance comparable to or even better than CNNs in the task of CD in recent years. 26,27,30,41,42 Chen et al 26 represented bitemporal images as semantic tokens and used transformers to model the context and feed it back into pixel space to obtain more refined original features. Bandara et al 27 proposed a Siamese network using a hierarchical transformer encoder combined with a multilayer perceptron (MLP) decoder for CD tasks that can more effectively represent multiscale remote details.…”

Filtering Specialized Change in a Few-Shot Setting

“…Figure 2 shows an illustration of this process. A similar approach is also described by Liu et al [55], who use the labels of the semantic change detection dataset HRSCD [34] to only select cropland changes as their binary labels.…”