SCViT: A Spatial-Channel Feature Preserving Vision Transformer for Remote Sensing Image Scene Classification

Lv, Pengyuan; Wu, Wenjun; Zhong, Yanfei; Du, Fang; Zhang, Liangpei

doi:10.1109/tgrs.2022.3157671

Cited by 95 publications

(46 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then, Bashmal et al [44] proposed the Dataefficient image transformers (DeiT), a ViT-based model trained by knowledge distillation with fewer data, and proved that the performance of ViT was superior to the CNN-based method on the remote sensing datasets AID and NWPU-RESISC. In [45], SCViT is proposed to overcome the disadvantage that the original model can only capture global spatial features. By The Similarity Between Scene "Railway Station" and "Industrial Area".…”

Section: Related Workmentioning

confidence: 99%

Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

Wang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing images (RSIs) are characterized by complex spatial layouts and ground object structures. ViT can be a good choice for scene classification owing to the ability to capture long-range interactive information between patches of input images. However, due to the lack of some inductive biases inherent to CNNs, such as locality and translation equivariance, ViT cannot generalize well when trained on insufficient amounts of data. Compared with training ViT from scratch, transferring a large-scale pre-trained one is more cost-efficient with better performance even when the target data is small-scale. In addition, In addition, the cross-entropy (CE) loss is frequently utilized in scene classification yet has low robustness to noise labels and poor generalization performances for different scenes. In this paper, a ViT-based model in combination with supervised contrastive learning (CL) is proposed, named ViT-CL. For CL, Supervised contrastive (SupCon) loss which is developed by extending the self-supervised contrastive approach to the fullysupervised setting, can explore the label information of RSIs in embedding space and improve the robustness to common image corruption. In ViT-CL, a joint loss function that combines CE loss and SupCon loss is developed to prompt the model to learn more discriminative features. Also, a two-stage optimization framework is introduced to enhance the controllability of the optimization process of the ViT-CL model. Extensive experiments on the AID, NWPU-RESISC45, and UCM datasets verified the superior performance of ViT-CL, with the highest accuracies of 97.42%, 94.54%, and 99.76% among all competing methods, respectively.

show abstract

Section: Related Workmentioning

confidence: 99%

Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

Wang

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…LGCNet [33] is the first CNN-based remote sensing image SR model that utilizes local and global representations to learn image residuals between HR images and upscaled LR images. SCViT [34] proposes a spatial channel feature preservation model that considers the detailed geometric information of the high-spatial-resolution imagery. TransENet [35] employs a multiscale transformer to aggregate multidimensional spatial features while focusing on image spatial self-similarity.…”

Section: B Lightweight Srmentioning

confidence: 99%

DARN: Distance Attention Residual Network for Lightweight Remote-Sensing Image Superresolution

Wang

Chen

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The application of single-image superresolution (SISR) in remote sensing is of great significance. Although the state-of-the-art convolution neural network (CNN)-based SISR methods have achieved excellent results, the large model and slow speed make it difficult to deploy in real remote sensing tasks. In this article, we propose a compact and efficient distance attention residual network (DARN) to achieve a better compromise between model accuracy and complexity. The distance attention residual connection block (DARCB), the core component of the DARN, uses multistage feature aggregation to learn more accurate feature representations. The main branch of the DARCB adopts a shallow residual block (SRB) to flexibly learn residual information to ensure the robustness of the model. We also propose a distance attention block (DAB) as a bridge between the main branch and the branch of the DARCB; the DAB can effectively alleviate the loss of detail features in the deep CNN extraction process. Experimental results on two remote sensing and five super-resolution benchmark datasets demonstrate that the DARN achieves a better compromise than existing methods in terms of performance and model complexity. In addition, the DARN achieves the optimal solution compared with the state-of-the-art lightweight remote sensing SISR method in terms of parameter amount, computation amount, and inference speed. Our code will be available at https://github.com/candygogogogo/DARN.

show abstract

“…In this subsection, we will introduce the swin transformer architecture, which is the backbone of Meta-TR network. Transformer was originally used in NLP (natural language processing) [42], and it has also shown its superiority in remote sensing image processing recent years [43]- [45]. However, the original transformer needs to pay attention to all pixels of image in the calculation, which leads to a sharp increase in calculation and increases the restrictions on deployment and application.…”

Section: B Swin Transformer Architecturementioning

confidence: 99%

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Cui

Yang

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

As a flourishing research topic in the field of remote sensing, SCI (spatial compressive imaging) can utilize prior knowledge to recover high-dimensional signals from LR (lowresolution) measurements through joint sampling and compression, thus contributing to the bandwidth reduction of information transmission. However, most of the existing SCI methods based on deep learning cannot effectively utilize prior information, and difficult to perform deep extraction of image features, so the reconstruction is not ideal in the case of low sampling ratio. To address the above difficulty, we propose a SCI network based on MA (meta-attention) and swin transformer, named Meta-TR. We adopt the swin transformer as the network backbone, through the wide application of self-attention mechanisms, to achieve deeper extraction of image features, thereby improving the reconstruction quality under low sampling ratios. In addition, we design a meta-attention module, which adopts Squeeze-Excitation architecture to convert the metadata of SCI image degradation process to attention vectors. Then the attention vectors are used in the channel modulation of network feature maps to guide the network training. Extensive experiments are performed on different benchmark remote sensing datasets and different sampling ratios to confirm the superiority of the proposed Meta-TR method.

show abstract

SCViT: A Spatial-Channel Feature Preserving Vision Transformer for Remote Sensing Image Scene Classification

Cited by 95 publications

References 46 publications

Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

DARN: Distance Attention Residual Network for Lightweight Remote-Sensing Image Superresolution

Meta-TR: Meta-Attention Spatial Compressive Imaging Network With Swin Transformer

Contact Info

Product

Resources

About