Change Detection in Hyperspectral Images Using Recurrent 3D Fully Convolutional Networks

Abstract

Hyperspectral change detection (CD) can be effectively performed using deep-learning networks. Although these approaches require qualified training samples, it is difficult to obtain ground-truth data in the real world. Preserving spatial information during training is difficult due to structural limitations. To solve such problems, our study proposed a novel CD method for hyperspectral images (HSIs), including sample generation and a deep-learning network, called the recurrent three-dimensional (3D) fully convolutional network (Re3FCN), which merged the advantages of a 3D fully convolutional network (FCN) and a convolutional long short-term memory (ConvLSTM). Principal component analysis (PCA) and the spectral correlation angle (SCA) were used to generate training samples with high probabilities of being changed or unchanged. The strategy assisted in training fewer samples of representative feature expression. The Re3FCN was mainly comprised of spectral-spatial and temporal modules. Particularly, a spectral-spatial module with a 3D convolutional layer extracts the spectral-spatial features from the HSIs simultaneously, whilst a temporal module with ConvLSTM records and analyzes the multi-temporal HSI change information. The study first proposed a simple and effective method to generate samples for network training. This method can be applied effectively to cases with no training samples. Re3FCN can perform end-to-end detection for binary and multiple changes. Moreover, Re3FCN can receive multi-temporal HSIs directly as input without learning the characteristics of multiple changes. Finally, the network could extract joint spectral-spatial-temporal features and it preserved the spatial structure during the learning process through the fully convolutional structure. This study was the first to use a 3D FCN and a ConvLSTM for the remote-sensing CD. To demonstrate the effectiveness of the proposed CD method, we performed binary and multi-class CD experiments. Results revealed that the Re3FCN outperformed the other conventional methods, such as change vector analysis, iteratively reweighted multivariate alteration detection, PCA-SCA, FCN, and the combination of 2D convolutional layers-fully connected LSTM. from various platforms, such as aircraft, satellites, and unmanned aerial vehicles [1]. The spectral profiles obtained from HSIs help to achieve target detection, classification, as well as change detection (CD) because of the profiles ability to distinguish the spectrally similar materials and describe the finer spectral changes [2,3].CD is the process of identifying changes in land cover or land use that have occurred over time in the same geographical area [4]. Applications of CD techniques include assessing natural disasters, monitoring crops, and managing water resources [5][6][7]. Moreover, CD has offered the benefit of hyperspectral remote sensing over the past two decades. As HSIs offer more-detailed information about spectral changes than multispectral images, HSIs can improve the performance of CD [8]. Ho...

Urban Building Change Detection in SAR Images Using Combined Differential Image and Residual U-Net Network

“…The FCN could extract more hierarchical features at different convolution layers due to its skip connection operation, which could combine low-level details features (like line, edge, and texture features) with high-level semantic features (class or context information). In addition, this network was more effective than the traditional CNN or DBN network in remote sensing classification [29][30][31] or change detection [32,33]. Its up-sampling layer could output…”

Methods and Challenges Using Multispectral and Hyperspectral Images for Practical Change Detection Applications

“…The following few papers [82][83][84] are related to deep learning. A paper by Lyu et al [82] was the first on the application of deep-learning-based recurrent neural networks for change detection.…”

“…Song et al [83] present a change detection framework for hyperspectral images using recurrent convolutional neural networks. The first step is the use of PCA to help select training samples.…”

Deep Quadruplet Network for Hyperspectral Image Classification with a Small Number of Samples

“…Since the concept of deep learning was introduced into hyperspectral image classification for the first time [10], deep neural network has been gaining popularity and has triggered global research interest in establishing deep learning models for hyperspectral image classification [11][12][13][14]. In particular, some deep-learning methods have been proposed by combining spectral and spatial features to improve classification accuracy [15][16][17][18][19][20][21].…”