Generative Adversarial Networks for Hyperspectral Image Classification

Abstract

A Generative Adversarial Network (GAN) usually contains a generative network and a discriminative network in competition with each other. The GAN has shown their capability in a variety of applications. In this paper, the usefulness and effectiveness of GAN for classification of hyperspectral images (HSIs) is explored for the first time. In the proposed GAN, a convolutional neural network (CNN) is designed to discriminate the inputs and another CNN is used to generate so-called fake inputs. The aforementioned CNNs are trained together: The generative CNN tries to generate fake inputs that are as real as possible, and the discriminative CNN tries to classify the real and fake inputs. This kind of adversarial training improves the generalization capability of the discriminative CNN, which is really important when the training samples are limited. Specifically, we propose two schemes: (1) a well-designed 1D-GAN as a spectral classifier, and (2) a robust 3D-GAN is as a spectral-spatial classifier. Furthermore, the generated adversarial samples are used with real training samples to fine-tune the discriminative CNN, which improves the final classification performance. The proposed classifiers are carried out on three widely-used hyperspectral datasets: Salinas, Indiana Pines, and Kennedy Space Center. The obtained results reveal that the proposed models provide competitive results compared to the state-of-the-art methods. In addition, the proposed GANs open new opportunities in the remote sensing community for the challenging task of HSIs classification, and also, reveal the huge potential of GAN-based methods for the analysis of such complex and inherently nonlinear data.

AeroRIT: A New Scene for Hyperspectral Image Analysis

“…Along with the promising performance that CNNs have achieved in the hyperspectral domain [35], [15], [16], [17], there are several directions of research that can be construed as open problems on this dataset: 1) Radiance or Reflectance: We compared the performance of U-Net-m-SE on the radiance and reflectance sets of images and obtain an mIOU of nearly 5 points less when using reflectance (reflectance-mIOU: 69.90 vs radiance-mIOU: 75.35).…”

“…Hao et al designed a twostream architecture, where stream1 used a stacked denoising autoencoder to encode the spectral values of each input pixel of a patch and stream2 used a CNN to process the patch's spatial features [15]. Zhu et al used a generative adversarial networks (GANs) to create robust classifiers of hyperspectral signatures [16]. Recently, Roy et al proposed using a 3D-CNN followed by a 2D-CNN to learn better abstract level representations for HSI scenes [17].…”

How can Big Data and machine learning benefit environment and water management: a survey of methods, applications, and future directions

“…They showed that their 3D CNN models were able to achieve a better classification rate than the standard 2D CNN models. He et al [156] and Zhu et al [157] applied GAN to HSI classification, who showed that GAN gave better performance than the traditional CNN when training data was limited.…”

DC‐AL GAN: Pseudoprogression and true tumor progression of glioblastoma multiform image classification based on DCGAN and AlexNet

“…A generative adversarial network (GAN) is a deep learning architecture in which two neural networks compete against each other in a zero‐sum game framework . Specifically, a GAN model consists of two parts: a generator and a discriminator.…”

“…A generative adversarial network (GAN) 43 is a deep learning architecture in which two neural networks compete against each other in a zero-sum game framework. [44][45][46][47][48][49] Specifically, a GAN model consists of two parts: a generator and a discriminator. In the training stage of a GAN model, the samples produced by the generator, together with real images, serve as inputs to the discriminator.…”