CFRWD-GAN for SAR-to-Optical Image Translation

Abstract

Synthetic aperture radar (SAR) images have been extensively used in earthquake monitoring, resource survey, agricultural forecasting, etc. However, it is a challenge to interpret SAR images with severe speckle noise and geometric deformation due to the nature of radar imaging. The translation of SAR-to-optical images provides new support for the interpretation of SAR images. Most of the existing translation networks, which are based on generative adversarial networks (GANs), are vulnerable to part information loss during the feature reasoning stage, making the outline of the translated images blurred and semantic information missing. Aiming to solve these problems, cross-fusion reasoning and wavelet decomposition GAN (CFRWD-GAN) is proposed to preserve structural details and enhance high-frequency band information. Specifically, the cross-fusion reasoning (CFR) structure is proposed to preserve high-resolution, detailed features and low-resolution semantic features in the whole process of feature reasoning. Moreover, the discrete wavelet decomposition (WD) method is adopted to handle the speckle noise in SAR images and achieve the translation of high-frequency components. Finally, the WD branch is integrated with the CFR branch through an adaptive parameter learning method to translate SAR images to optical ones. Extensive experiments conducted on two publicly available datasets, QXS-SAROPT and SEN1-2, demonstrate a better translation performance of the proposed CFRWD-GAN compared to five other state-of-the-art models.

SAR-to-Optical Image Translation via an Interpretable Network

“…By leveraging the provided information such as class labels in the generator of the original GAN, Conditional Generative Adversarial Networks (cGANs) [1] can learn an adaptive sample generation process for different classes during training, thus enabling the model to generate more realistic images belonging to a given class. However, for the S2O image conversion task, the generated optical images usually do not have the desired fine structure since there is a substantial inter-domain gap existing between the SAR image and the optical image [49][50][51][52]. Isola et al presented a pix2pix [53] model based on a cGAN and utilized the input image as a condition for image translation.…”