Hyperspectral Image Classification Method Based on 2D–3D CNN and Multibranch Feature Fusion

Ge, Zixian; Cao, Guo; Li, Xuesong; Fu, Peng

doi:10.1109/jstars.2020.3024841

Cited by 109 publications

(45 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In order to evaluate the classification performance of the proposed HSMSN-HFF, the performance of six state-of-the-art DL-based methods is given in this section to compare with the HSMSN-HFF. The six methods are: 3D-CNN [26], DFFN [43], MSDN [59], HybirdSN [35], MDR-CNN [55], and 2D3D-MBFF [60]. 3D-CNN is a shallow CNN model, which is constructed with two 3D Conv-pooling blocks and classified by logistic regression.…”

Section: E Comparison Results Of Different Methodsmentioning

confidence: 99%

Hierarchical Shrinkage Multiscale Network for Hyperspectral Image Classification With Hierarchical Feature Fusion

Chen

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Recently, deep learning (DL) based hyperspectral image classification (HSIC) has attracted substantial attention.Many works based on the convolutional neural network (CNN) model have been certificated to be significantly successful for boosting the performance of HSIC. However, most of these methods extract features by using a fixed convolutional kernel and ignore multi-scale features of the ground objects of hyperspectral images (HSIs). Although some recent methods have proposed multi-scale feature extraction schemes, more computing and storage resources were consumed. Moreover, when using CNN to implement hyperspectral image classification, many methods only use the high-level semantic information extracted from the end of the network, ignoring the edge information extracted from shallow layers of the network. To settle the preceding two issues, a novel HSIC method based on hierarchical shrinkage multi-scale network (HSMSN) and the hierarchical feature fusion (HFF) is proposed, with which the newly proposed classification framework can fuse features generated by both of multi-scale receptive field and multiple levels. Specifically, multi-depth and multi-scale residual block (MDMSRB) is constructed by superposition dilated convolution to realize multi-scale feature extraction. Furthermore, according to the change of feature size in different stages of the neural networks, we design a hierarchical shrinkage multi-scale feature extraction network by pruning MDMSRB to reduce the redundancy of network structure. In addition, to make full use of the features extracted in each stage of the network, the proposed network hierarchically integrates low-level edge features and high-level semantic features effectively. Experimental results demonstrate that the proposed method achieves more competitive performance with a limited computational cost than other state-of-the-art methods. Index Terms-Hyperspectral image classification (HSIC), convolutional neural network (CNN), hierarchical shrinkage multi-scale network (HSMSN), multi-depth and multi-scale residual block (MDMSRB), hierarchical feature fusion (HFF)Ⅰ. INTRODUCTION ith the rapid development of imaging spectrometers and platforms, imaging spectroscopy (also called This paragraph of the first footnote will contain the date on which you submitted your paper for review.

show abstract

Section: E Comparison Results Of Different Methodsmentioning

confidence: 99%

Hierarchical Shrinkage Multiscale Network for Hyperspectral Image Classification With Hierarchical Feature Fusion

Chen

2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In [27], a novel supervised deep feature extraction algorithm combined siamese CNN with linear SVM was introduced. Chen et al [28] [29]- [32]. It also has been verified that the spatial correlation across HSIs can provide complementary information to spectral features and should be taken into account [33]- [39].…”

Section: Introductionmentioning

confidence: 94%

A Pixel Cluster CNN and Spectral-Spatial Fusion Algorithm for Hyperspectral Image Classification With Small-Size Training Samples

Dong

Quan

Feng

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Convolutional neural networks (CNNs) can automatically learn features from the hyperspectral image (HSI) data, avoiding the difficulty of manually extracting features. However, the number of training samples for the classification of HSIs is always limited, making it difficult for CNN to obtain effective features and resulting in low classification accuracy. To solve this problem, a pixel cluster CNN and spectral-spatial fusion (PC-CNN-SSF) algorithm for hyperspectral image classification (HSIC) with small-size training samples is proposed in this paper. Firstly, spatial information is extracted by the Gray Level Co-occurrence Matrix (GLCM). Then spatial information and spectral information are fused by means of bands superposition, forming spectral-spatial features. To expand the number of training samples, the pixels after spectral-spatial fusion are combined into pixel clusters according to a certain rule. Finally, a CNN framework is utilized to extract effective features from the pixel clusters. Experiments based on three standard HSIs demonstrate that the proposed algorithm can get better performance than the conventional CNN and also outperforms other studied algorithms in the case of small training set.

show abstract

“…On this basis, Feng et al [38] proposed a R-HybridSN network with skip connections and depth-separable convolution and achieved better classification results than all the contrast models using very few training samples. Ge et al [39] also proved the effectiveness of 2D-3D CNN with multibranch feature fusion.…”

Section: Introductionmentioning

confidence: 97%

Multiscale Information Fusion for Hyperspectral Image Classification Based on Hybrid 2D-3D CNN

Gong

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

Hyperspectral images are widely used for classification due to its rich spectral information along with spatial information. To process the high dimensionality and high nonlinearity of hyperspectral images, deep learning methods based on convolutional neural network (CNN) are widely used in hyperspectral classification applications. However, most CNN structures are stacked vertically in addition to using a onefold size of convolutional kernels or pooling layers, which cannot fully mine the multiscale information on the hyperspectral images. When such networks meet the practical challenge of a limited labeled hyperspectral image dataset—i.e., “small sample problem”—the classification accuracy and generalization ability would be limited. In this paper, to tackle the small sample problem, we apply the semantic segmentation function to the pixel-level hyperspectral classification due to their comparability. A lightweight, multiscale squeeze-and-excitation pyramid pooling network (MSPN) is proposed. It consists of a multiscale 3D CNN module, a squeezing and excitation module, and a pyramid pooling module with 2D CNN. Such a hybrid 2D-3D-CNN MSPN framework can learn and fuse deeper hierarchical spatial–spectral features with fewer training samples. The proposed MSPN was tested on three publicly available hyperspectral classification datasets: Indian Pine, Salinas, and Pavia University. Using 5%, 0.5%, and 0.5% training samples of the three datasets, the classification accuracies of the MSPN were 96.09%, 97%, and 96.56%, respectively. In addition, we also selected the latest dataset with higher spatial resolution, named WHU-Hi-LongKou, as the challenge object. Using only 0.1% of the training samples, we could achieve a 97.31% classification accuracy, which is far superior to the state-of-the-art hyperspectral classification methods.

show abstract

Hyperspectral Image Classification Method Based on 2D–3D CNN and Multibranch Feature Fusion

Cited by 109 publications

References 61 publications

Hierarchical Shrinkage Multiscale Network for Hyperspectral Image Classification With Hierarchical Feature Fusion

Hierarchical Shrinkage Multiscale Network for Hyperspectral Image Classification With Hierarchical Feature Fusion

A Pixel Cluster CNN and Spectral-Spatial Fusion Algorithm for Hyperspectral Image Classification With Small-Size Training Samples

Multiscale Information Fusion for Hyperspectral Image Classification Based on Hybrid 2D-3D CNN

Contact Info

Product

Resources

About