Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks

Fu, Shilei; Xu, Feng; Jin, Ya‐Qiu

doi:10.1007/s11432-020-3077-5

Cited by 46 publications

(9 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[24] improved results by including Residual Blocks [19] in the generator. Similarly, authors from [25] introduced cascaded-residual connections between the input and each U-Net [18] deconvolution stage to reach a more qualitative high resolution translated image.…”

Section: Related Workmentioning

confidence: 99%

Deep Learning of Radiometrical and Geometrical Sar Distorsions for Image Modality translations

Bralet

Atto

Chanussot

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Multimodal approaches for Earth Observations suffer from both the lack of interpretability of SAR images and the high sensitivity to meteorological conditions of optical images. Translation methods were implemented to solve them for specific tasks and areas. But these implementations lack of generalizability as they do not include samples with challenging characteristics. Firstly, this paper sums up the main problems that a general SAR to optical image translator should overcome. Then, a SAR Distorted Image to optical translator Network (SARDINet) alternating knowledgeable channel-wise spatial convolutions and cross-channel convolutions is implemented. It aims at solving a problem of major concern in remote sensing: translating layover disturbed SAR images into disturbance-free optical ones. SARDINet is trained through a classical and an adversarial framework and compared to cGAN and cycleGAN from the literature. Experimental results prove that adversarial approaches are more qualitative but worsen quantitative results.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Learning of Radiometrical and Geometrical Sar Distorsions for Image Modality translations

Bralet

Atto

Chanussot

et al. 2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

show abstract

“…Researchers are now blending the strengths of both Pix2Pix and cycleGAN to enhance SAR-to-optical image translation, focusing on improvements in generators, discriminators, loss functions, and overall model structure. Improvements in generators often involve residual modules, dense connections, and dilated convolutions, aiming to enhance the interaction between encoders and decoders and preserve intricate details like color and texture (Ley et al, 2018 ; Turnes et al, 2020 ; Darbaghshahi et al, 2021 ; Fu et al, 2021 ; Guo et al, 2022 ; Shi et al, 2022 ; Yang et al, 2022 ). On the discriminator front, using multiple discriminators at different scales helps more accurately distinguish between translated and real images (Guo et al, 2022 ; Yang et al, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

“…In terms of loss functions, similarity metrics and perceptual loss are gaining traction (Hwang et al, 2020 ; Li et al, 2020 ; Darbaghshahi et al, 2021 ). As for the model structure, some studies also explore bidirectional generators, combining elements from Pix2Pix and cycleGAN, to enhance the quality of translated images (Wang et al, 2019 ; Fu et al, 2021 ; Shi et al, 2022 ). Despite these advancements, challenges persist due to the inherent differences between SAR and optical images.…”

Section: Introductionmentioning

confidence: 99%

A brain-inspired approach for SAR-to-optical image translation based on diffusion models

Shi,

Cui,

Chen

et al. 2024

Front. Neurosci.

View full text Add to dashboard Cite

Synthetic Aperture Radar (SAR) plays a crucial role in all-weather and all-day Earth observation owing to its distinctive imaging mechanism. However, interpreting SAR images is not as intuitive as optical images. Therefore, to make SAR images consistent with human cognitive habits and assist inexperienced people in interpreting SAR images, a generative model is needed to realize the translation from SAR images to optical ones. In this work, inspired by the processing of the human brain in painting, a novel conditional image-to-image translation framework is proposed for SAR to optical image translation based on the diffusion model. Firstly, considering the limited performance of existing CNN-based feature extraction modules, the model draws insights from the self-attention and long-skip connection mechanisms to enhance feature extraction capabilities, which are aligned more closely with the memory paradigm observed in the functioning of human brain neurons. Secondly, addressing the scarcity of SAR-optical image pairs, data augmentation that does not leak the augmented mode into the generated mode is designed to optimize data efficiency. The proposed SAR-to-optical image translation method is thoroughly evaluated using the SAR2Opt dataset. Experimental results demonstrate its capacity to synthesize high-fidelity optical images without introducing blurriness.

show abstract

“…The exponential growth of remote sensing technology [1][2][3][4] has made high-quality remote sensing images (RSIs) acquisition more accessible and cost-affordable. High resolution is a crucial characteristic of RSIs, with spatial resolutions generally ranging between 5 and 10 cm.…”

Section: Introductionmentioning

confidence: 99%

EAD-Net: Efficiently Asymmetric Network for Semantic Labeling of High-Resolution Remote Sensing Images with Dynamic Routing Mechanism

Hu,

Wang,

2024

Remote Sensing

View full text Add to dashboard Cite

Semantic labeling of high-resolution remote sensing images (HRRSIs) holds a significant position in the remote sensing domain. Although numerous deep-learning-based segmentation models have enhanced segmentation precision, their complexity leads to a significant increase in parameters and computational requirements. While ensuring segmentation accuracy, it is also crucial to improve segmentation speed. To address this issue, we propose an efficient asymmetric deep learning network for HRRSIs, referred to as EAD-Net. First, EAD-Net employs ResNet50 as the backbone without pooling, instead of the RepVGG block, to extract rich semantic features while reducing model complexity. Second, a dynamic routing module is proposed in EAD-Net to adjust routing based on the pixel occupancy of small-scale objects. Concurrently, a channel attention mechanism is used to preserve their features even with minimal occupancy. Third, a novel asymmetric decoder is introduced, which uses convolutional operations while discarding skip connections. This not only effectively reduces redundant features but also allows using low-level image features to enhance EAD-Net’s performance. Extensive experimental results on the ISPRS 2D semantic labeling challenge benchmark demonstrate that EAD-Net achieves state-of-the-art (SOTA) accuracy performance while reducing model complexity and inference time, while the mean Intersection over Union (mIoU) score reaching 87.38% and 93.10% in the Vaihingen and Potsdam datasets, respectively.

show abstract

Reciprocal translation between SAR and optical remote sensing images with cascaded-residual adversarial networks

Cited by 46 publications

References 16 publications

Deep Learning of Radiometrical and Geometrical Sar Distorsions for Image Modality translations

Deep Learning of Radiometrical and Geometrical Sar Distorsions for Image Modality translations

A brain-inspired approach for SAR-to-optical image translation based on diffusion models

EAD-Net: Efficiently Asymmetric Network for Semantic Labeling of High-Resolution Remote Sensing Images with Dynamic Routing Mechanism

Contact Info

Product

Resources

About