Self-Supervised Pansharpening Based on a Cycle-Consistent Generative Adversarial Network

Li, Jie; Sun, Weixuan; Jiang, Menghui; Yuan, Qiangqiang

doi:10.1109/lgrs.2021.3137428

Cited by 15 publications

(13 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, the medical images collected by instruments with different parameters have certain deviations. Furthermore, this paper uses the Evaluation indicators RIED-Net [5] pix2pix [7] Reset GAN [11] pGAN [15] Residual U-Net GAN [17] Proposed algorithm same preprocessing steps to process all data. Therefore, the data collection method and preprocessing method will have a particular impact on the experimental results.…”

Section: Discussionmentioning

confidence: 99%

“…Literature [ 4 ] suggested a context-aware generative adversarial network that uses an artificial context model to get the high-accuracy and resilient mapping from MRI to CT (computed tomography) images. A multichannel generative adversarial network was presented in the literature [ 5 ] to manufacture PET pictures. The experiment was carried out on 50 lung cancer patients' PET-CT data to produce more realistic PET pictures.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

CNN-Based Cross-Modal Residual Network for Image Synthesis

Kumar

Bhatnagar

Jain

et al. 2022

BioMed Research International

View full text Add to dashboard Cite

This study attempts to address the issue that present cross-modal image synthesis algorithms do not capture the spatial and structural information of human tissues effectively. As a consequence, the resulting photos include flaws including fuzzy edges and a poor signal-to-noise ratio. The authors offer a cross-sectional technique that combines residual modules with generative adversarial networks. The approach incorporates an enhanced residual initial module and attention mechanism into the generator network, reducing the number of parameters and improving the generator’s feature learning capabilities. To boost discriminant performance, the discriminator employs a multiscale discriminator. A multilevel structural similarity loss is included in the loss function to improve picture contrast preservation. On the ADNI data set, the algorithm is compared to the mainstream algorithms. The experimental findings reveal that the synthetic PET image’s MAE index has dropped while the SSIM and PSNR indexes have improved. The experimental findings suggest that the proposed model may maintain picture structural information while improving image quality in both visual and objective measures. The residue initial module and attention mechanism are employed to increase the generator’s capacity for learning, while the multiscale discriminator is utilized to improve the model’s discriminative performance. The enhanced method in this study can maintain the structure and contrast information of the picture, according to comparative experimental findings using the ADNI dataset. The produced picture is hence more aesthetically similar to the genuine print.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

CNN-Based Cross-Modal Residual Network for Image Synthesis

Kumar

Bhatnagar

Jain

et al. 2022

BioMed Research International

View full text Add to dashboard Cite

show abstract

“…Ozcelik et al [32] constructed a selfsupervised learning framework considering pansharpening as a colorization, i.e., PanColorGAN, which reduces blurring by color injection and random-scale downsampling. Li et al [33] put forward a self-supervised approach using a cycleconsistent GAN trained on reduced resolution data, which builds two generators and two discriminators. The LRMS and PAN images are fed into the first generator to yield the predicted image, and then the predicted image is input to the second generator to acquire the PAN image, which remains consistent with the input PAN.…”

Section: Introductionmentioning

confidence: 99%

Pansharpening Using Unsupervised Generative Adversarial Networks With Recursive Mixed-Scale Feature Fusion

Feng

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Panchromatic sharpening (pansharpening) is an important technology for improving the spatial resolution of multispectral (MS) images. The majority of the models are implemented at the reduced resolution, leading to unfavorable results at the full resolution. Moreover, the complicated relationship between MS and panchromatic (PAN) images is often ignored in detail injection. For the mentioned problems, unsupervised generative adversarial networks with recursive mixed-scale feature fusion for pansharpening (RMFF-UPGAN) are modeled to boost the spatial resolution and preserve the spectral information. RMFF-UPGAN comprises a generator and two U-shaped discriminators. A dual-stream trapezoidal branch is designed in the generator to obtain multiscale information. Further, a recursive mixed-scale feature fusion subnetwork is designed. Perform a prior fusion on the extracted MS and PAN features of the same scale. A mixed-scale fusion is conducted on the prior fusion results of the fine-scale and coarse-scale. The fusion is executed sequentially in the above manner building a recursive mixed-scale fusion structure and finally generating key information. A compensation information mechanism is also designed for the reconstruction of key information to compensate for information. A nonlinear rectification block for the reconstructed information is developed to overcome the distortion induced by neglecting the complicated relationship between MS and PAN images. Two U-shaped discriminators are designed and a new composite loss function is defined. The presented model is validated using two satellite data and the outcomes reveal better than the prevalent approaches regarding both visual assessment and objective indicators.

show abstract

“…In this method, multiscale generators were constructed to enrich the spatial details in the fused image progressively. Li et al [38] employed a cycle-consistent GAN to achieve unsupervised training on unpaired datasets. Furthermore, recent networks based on the transformer are drawing the attention of researchers.…”

Section: Introductionmentioning

confidence: 99%

Spatial-Spectral Dual Back-Projection Network for Pansharpening

Zhang

Wang

Zhang

et al. 2023

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

Deep unfolding networks have obtained satisfactory performance in the pansharpening task owing to their sufficient interpretability. Inspired by the back-projection (BP) mechanism, we propose a BP-driven model, spatial-spectral dual back-project network (S 2 DBPN), to fuse the low spatial resolution multispectral (LR MS) and the high spatial resolution panchromatic (PAN) images by exploiting the BP in spatial and spectral domains. Specifically, the proposed S 2 DBPN is made up of a spatial BP network, a spectral BP network, and a reconstruction network. In the spatial BP network, spatial down-and up-projection modules are derived from BP, which is responsible for the projection of the LR MS image into the spatial domain. By analogy with the spatial BP, we reformulate the degradation between high spatial resolution multispectral (HR MS) and PAN images as spectral down-and up-projections. Then, the spectral BP network is constructed for the projection of the PAN image along the channel dimension. Finally, the features from spatial and spectral BP networks are integrated to produce the desired HR MS image through the reconstruction network. Compared to the state-of-the-art methods, extensive experiments on QuickBird, GeoEye-1, and WorldView-2 datasets demonstrate that our S 2 DBPN produces better HR MS images in terms of qualitative and quantitative evaluation metrics. The code of S 2 DBPN is released at: https://github.com/RSMagneto/S2DBPN.

show abstract

Self-Supervised Pansharpening Based on a Cycle-Consistent Generative Adversarial Network

Cited by 15 publications

References 12 publications

CNN-Based Cross-Modal Residual Network for Image Synthesis

CNN-Based Cross-Modal Residual Network for Image Synthesis

Pansharpening Using Unsupervised Generative Adversarial Networks With Recursive Mixed-Scale Feature Fusion

Spatial-Spectral Dual Back-Projection Network for Pansharpening

Contact Info

Product

Resources

About