PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

Liu, Qingjie; Zhou, Huanyu; Xu, Qizhi; Liu, Xiangyu; Wang, Yunlong

doi:10.1109/tgrs.2020.3042974

Cited by 192 publications

(118 citation statements)

References 52 publications

Supporting

Mentioning

118

Contrasting

Order By: Relevance

“…In our future work, we will continue to study the architecture of unsupervised pan-sharpening models and further improve the performance. (d) GS [5] (e) IHS [2] (f) Brovey [47] (g) HPF [48] (h) LMM [49] (i) SFIM [8] (j) PNN [15] (k) DRPNN [19] (l) MSDCNN [20] (m) PanNet [17] (n) PSGAN [23] (o) Pan-GAN [40] (p) PGMAN (d) GS [5] (e) IHS [2] (f) Brovey [47] (g) HPF [48] (h) LMM [49] (i) SFIM [8] (j) PNN [15] (k) DRPNN [19] (l) MSDCNN [20] (m) PanNet [17] (n) PSGAN [23] (o) Pan-GAN [40] (p) PGMAN…”

Section: Discussionmentioning

confidence: 99%

“…Supervised methods are PNN [15], DRPNN [19], MSD-CNN [20], PanNet [17] and PSGAN [23]. These methods are state-of-the-art supervised deep learning based pan-sharpening methods.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

“…TFNet [21] builds a two-stream fusion network and designs a variant of UNet [22] to solve the problem. PSGAN [23] improves the TFNet [21] by using generative adversarial training [24].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

PGMAN: An Unsupervised Generative Multiadversarial Network for Pansharpening

Zhou

Liu

Wang

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

Pan-sharpening aims at fusing a low-resolution (LR) multi-spectral (MS) image and a high-resolution (HR) panchromatic (PAN) image acquired by a satellite to generate an HR MS image. Many deep learning based methods have been developed in the past few years. However, since there are no intended HR MS images as references for learning, almost all of the existing methods down-sample the MS and PAN images and regard the original MS images as targets to form a supervised setting for training. These methods may perform well on the down-scaled images, however, they generalize poorly to the full-resolution images. To conquer this problem, we design an unsupervised framework that is able to learn directly from the full-resolution images without any preprocessing. The model is built based on a novel generative multi-adversarial network. We use a twostream generator to extract the modality-specific features from the PAN and MS images, respectively, and develop a dualdiscriminator to preserve the spectral and spatial information of the inputs when performing fusion. Furthermore, a novel loss function is introduced to facilitate training under the unsupervised setting. Experiments and comparisons with other stateof-the-art methods on GaoFen-2, QuickBird and WorldView-3 images demonstrate that the proposed method can obtain much better fusion results on the full-resolution images. Code is available 1 .

show abstract

Section: Discussionmentioning

confidence: 99%

“…Supervised methods are PNN [15], DRPNN [19], MSD-CNN [20], PanNet [17] and PSGAN [23]. These methods are state-of-the-art supervised deep learning based pan-sharpening methods.…”

Section: Comparison With State-of-the-artsmentioning

confidence: 99%

See 1 more Smart Citation

PGMAN: An Unsupervised Generative Multiadversarial Network for Pansharpening

Zhou

Liu

Wang

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

show abstract

“…GANs are also promising RSIF models. In 2018, PSGAN proposed by Liu et al [36]. The authors first designed a two-stream fusion structure to generate a high-resolution MS image, and then used a full convolution network as a discriminator to distinguish the real and generated (fused) images.…”

Section: A Remote Sensing Image Fusion Based On Deep Learningmentioning

confidence: 99%

Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure

Jin

Huang

Jiang

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image fusion (RSIF) can generate an integrated image with high spatial and spectral resolution. The fused remote sensing image is conducive to applications including disaster monitoring, ecological environment investigation, and dynamic monitoring. However, most existing deep learning-based RSIF methods require ground truths (or reference images) to train a model, and the acquisition of ground truths is a difficult problem. To address this, we propose a semi-supervised RSIF method based on the multi-scale conditional generative adversarial networks (cGANs) by combining the multi-skip connection and pseudo-Siamese structure. This new method can simultaneously extract the features of panchromatic and multi-spectral images to fuse them without a ground truth; the adopted multi-skip connection contributes to presenting image details. In addition, we propose a composite loss function, which combines the least squares loss, L1 loss, and peak signal-noise ratio loss to train the model; the composite loss function can help to retain the spatial details and spectral information of the source images. Moreover, we verify the proposed method by extensive experiments, and the results show that the new method can achieve outstanding performance without relying on the ground truth. Index Terms-conditional generative adversarial network; deep learning; image fusion; loss function; remote sensing image fusion I. INTRODUCTION MAGE fusion aims to fuse the complementary information in two or more source images obtained by different sensors so that a new comprehensive image can be generated [1][2][3]. The fused image can present more useful information than any one of the source images. Therefore, image fusion techniques can provide a high-quality fused image to help interpret a scene or target, and related research areas include medical images [5], multi-focus images [3][6], infrared and visible images [7][8], and remote sensing images [9][10][10].Remote sensing images with high spatial and spectral resolution play an important role in geological exploration,

show abstract

“…However, PanNet's learning in the low-pass domain is still insufficient. The generative adversarial network (GAN) for remote sensing image pansharpening (PSGAN) generative adversarial network for remote sensing image pansharpening [19] is the first algorithm to use a GAN in pansharpening [20]. It designs a dual-stream CNN architecture for shallow feature extraction.…”

Section: Introductionmentioning

confidence: 99%

A Pansharpening Generative Adversarial Network with Multilevel Structure Enhancement and a Multistream Fusion Architecture

Zhang

Huang

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Deep learning has been widely used in various computer vision tasks. As a result, researchers have begun to explore the application of deep learning for pansharpening and have achieved remarkable results. However, most current pansharpening methods focus only on the mapping relationship between images and the lack overall structure enhancement, and do not fully and completely research optimization goals and fusion rules. Therefore, for these problems, we propose a pansharpening generative adversarial network with multilevel structure enhancement and a multistream fusion architecture. This method first uses multilevel gradient operators to obtain the structural information of the high-resolution panchromatic image. Then, it combines the spectral features with multilevel gradient information and inputs them into two subnetworks of the generator for fusion training. We design a comprehensive optimization goal for the generator, which not only minimizes the gap between the fused image and the real image but also considers the adversarial loss between the generator and the discriminator and the multilevel structure loss between the fused image and the panchromatic image. It is worth mentioning that we comprehensively consider the spectral information and the multilevel structure as the input of the discriminator, which makes it easier for the discriminator to distinguish real and fake images. Experiments show that our proposed method is superior to state-of-the-art methods in both the subjective visual and objective assessments of fused images, especially in road and building areas.

show abstract

PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

Cited by 192 publications

References 52 publications

PGMAN: An Unsupervised Generative Multiadversarial Network for Pansharpening

PGMAN: An Unsupervised Generative Multiadversarial Network for Pansharpening

Semisupervised Remote Sensing Image Fusion Using Multiscale Conditional Generative Adversarial Network With Siamese Structure

A Pansharpening Generative Adversarial Network with Multilevel Structure Enhancement and a Multistream Fusion Architecture

Contact Info

Product

Resources

About