Swin Transformer Embedding Dual-Stream for Semantic Segmentation of Remote Sensing Imagery

Zhou, Xuanyu; Zhou, Lifan; Gong, Shengrong; Zhong, Shan; Yan, Wei; Huang, Yizhou

doi:10.1109/jstars.2023.3326967

Cited by 12 publications

(2 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The output of one block is used as the input for the next, allowing the model to capture global dependencies across patches Zhou et al. ( 39 ). Finally, following hierarchical processing, a classification head is attached to the Swin Transformer to predict class labels for image classification tasks.…”

Section: Methodsmentioning

confidence: 99%

Swin-GA-RF: genetic algorithm-based Swin Transformer and random forest for enhancing cervical cancer classification

Alohali,

El-Rashidy,

Alaklabi

et al. 2024

Front. Oncol.

View full text Add to dashboard Cite

Cervical cancer is a prevalent and concerning disease affecting women, with increasing incidence and mortality rates. Early detection plays a crucial role in improving outcomes. Recent advancements in computer vision, particularly the Swin transformer, have shown promising performance in image classification tasks, rivaling or surpassing traditional convolutional neural networks (CNNs). The Swin transformer adopts a hierarchical and efficient approach using shifted windows, enabling the capture of both local and global contextual information in images. In this paper, we propose a novel approach called Swin-GA-RF to enhance the classification performance of cervical cells in Pap smear images. Swin-GA-RF combines the strengths of the Swin transformer, genetic algorithm (GA) feature selection, and the replacement of the softmax layer with a random forest classifier. Our methodology involves extracting feature representations from the Swin transformer, utilizing GA to identify the optimal feature set, and employing random forest as the classification model. Additionally, data augmentation techniques are applied to augment the diversity and quantity of the SIPaKMeD1 cervical cancer image dataset. We compare the performance of the Swin-GA-RF Transformer with pre-trained CNN models using two classes and five classes of cervical cancer classification, employing both Adam and SGD optimizers. The experimental results demonstrate that Swin-GA-RF outperforms other Swin transformers and pre-trained CNN models. When utilizing the Adam optimizer, Swin-GA-RF achieves the highest performance in both binary and five-class classification tasks. Specifically, for binary classification, it achieves an accuracy, precision, recall, and F1-score of 99.012, 99.015, 99.012, and 99.011, respectively. In the five-class classification, it achieves an accuracy, precision, recall, and F1-score of 98.808, 98.812, 98.808, and 98.808, respectively. These results underscore the effectiveness of the Swin-GA-RF approach in cervical cancer classification, demonstrating its potential as a valuable tool for early diagnosis and screening programs.

show abstract

Section: Methodsmentioning

confidence: 99%

Swin-GA-RF: genetic algorithm-based Swin Transformer and random forest for enhancing cervical cancer classification

Alohali,

El-Rashidy,

Alaklabi

et al. 2024

Front. Oncol.

View full text Add to dashboard Cite

show abstract

“…Remote sensing images have been widely applied in object detection [1,41], image semantic segmentation [42], image classification [43], and change detection [44]. However, the captured remote sensing images under diverse light conditions, such as overexposure and underexposure, often suffer from low dynamics and noise [45,46].…”

Section: Remote Sensing Image Enhancementmentioning

confidence: 99%

ERS-HDRI: Event-Based Remote Sensing HDR Imaging

Li,

Cheng,

Zeng

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

High dynamic range imaging (HDRI) is an essential task in remote sensing, enhancing low dynamic range (LDR) remote sensing images and benefiting downstream tasks, such as object detection and image segmentation. However, conventional frame-based HDRI methods may encounter challenges in real-world scenarios due to the limited information inherent in a single image captured by conventional cameras. In this paper, an event-based remote sensing HDR imaging framework is proposed to address this problem, denoted as ERS-HDRI, which reconstructs the remote sensing HDR image from a single-exposure LDR image and its concurrent event streams. The proposed ERS-HDRI leverages a coarse-to-fine framework, incorporating the event-based dynamic range enhancement (E-DRE) network and the gradient-enhanced HDR reconstruction (G-HDRR) network. Specifically, to efficiently achieve dynamic range fusion from different domains, the E-DRE network is designed to extract the dynamic range features from LDR frames and events and perform intra- and cross-attention operations to adaptively fuse multi-modal data. A denoise network and a dense feature fusion network are then employed for the generation of the coarse, clean HDR image. Then, the G-HDRR network, with its gradient enhancement module and multiscale fusion module, performs structure enforcement on the coarse HDR image and generates a fine informative HDR image. In addition, this work introduces a specialized hybrid imaging system and a novel, real-world event-based remote sensing HDRI dataset that contains aligned remote sensing LDR images, remote sensing HDR images, and concurrent event streams for evaluation. Comprehensive experiments have demonstrated the effectiveness of the proposed method. Specifically, it improves state-of-the-art PSNR by about 30% and the SSIM score by about 9% on the real-world dataset.

show abstract

Multi-scale fusion pixel and instance contrastive self-supervised learning for semantic segmentation of high-resolution Earth surface images

Liu,

Li,

2024

Advances in Space Research

View full text Add to dashboard Cite

Swin Transformer Embedding Dual-Stream for Semantic Segmentation of Remote Sensing Imagery

Cited by 12 publications

References 60 publications

Swin-GA-RF: genetic algorithm-based Swin Transformer and random forest for enhancing cervical cancer classification

Swin-GA-RF: genetic algorithm-based Swin Transformer and random forest for enhancing cervical cancer classification

ERS-HDRI: Event-Based Remote Sensing HDR Imaging

Multi-scale fusion pixel and instance contrastive self-supervised learning for semantic segmentation of high-resolution Earth surface images

Contact Info

Product

Resources

About