A Pansharpening Generative Adversarial Network with Multilevel Structure Enhancement and a Multistream Fusion Architecture

Abstract

Deep learning has been widely used in various computer vision tasks. As a result, researchers have begun to explore the application of deep learning for pansharpening and have achieved remarkable results. However, most current pansharpening methods focus only on the mapping relationship between images and the lack overall structure enhancement, and do not fully and completely research optimization goals and fusion rules. Therefore, for these problems, we propose a pansharpening generative adversarial network with multilevel structure enhancement and a multistream fusion architecture. This method first uses multilevel gradient operators to obtain the structural information of the high-resolution panchromatic image. Then, it combines the spectral features with multilevel gradient information and inputs them into two subnetworks of the generator for fusion training. We design a comprehensive optimization goal for the generator, which not only minimizes the gap between the fused image and the real image but also considers the adversarial loss between the generator and the discriminator and the multilevel structure loss between the fused image and the panchromatic image. It is worth mentioning that we comprehensively consider the spectral information and the multilevel structure as the input of the discriminator, which makes it easier for the discriminator to distinguish real and fake images. Experiments show that our proposed method is superior to state-of-the-art methods in both the subjective visual and objective assessments of fused images, especially in road and building areas.

An Assisted Teaching Method of College English Translation Using Generative Adversarial Network

“…Cheng et al put forward a text generation model based on GAN and TextGAN, which uses LSTM (long short-term merry network) as the generator and convolution network as the discriminator [11]. Zhang et al made a change in the generator because the binary output of the traditional GAN discriminator cannot transmit more meaningful information to the generator, so it cannot generate long text well [12]. Deng et al's research shows that the depth generation model has a more comprehensive feature expression ability.…”

Improved Generalized IHS Based on Total Variation for Pansharpening

“…The variational optimization-based techniques perform well in both spectral and spatial fidelity, with the drawback that they take more optimization iterations than the CS and MRA methods [14]. The last category is machine learning-based methods, where compressed perception-based [15] and dictionary-based [16] versions are the early algorithms, and CNN [17][18][19][20][21][22][23][24][25][26] and GAN [27][28][29][30][31] in deep learning are moving into the field of pansharpening with promising achievements. The pansharpening network (PNN) [32] has achieved encouraging results and a following from researchers.…”