3D Octave and 2D Vanilla Mixed Convolutional Neural Network for Hyperspectral Image Classification with Limited Samples

Abstract

Owing to the outstanding feature extraction capability, convolutional neural networks (CNNs) have been widely applied in hyperspectral image (HSI) classification problems and have achieved an impressive performance. However, it is well known that 2D convolution suffers from the absent consideration of spectral information, while 3D convolution requires a huge amount of computational cost. In addition, the cost of labeling and the limitation of computing resources make it urgent to improve the generalization performance of the model with scarcely labeled samples. To relieve these issues, we design an end-to-end 3D octave and 2D vanilla mixed CNN, namely Oct-MCNN-HS, based on the typical 3D-2D mixed CNN (MCNN). It is worth mentioning that two feature fusion operations are deliberately constructed to climb the top of the discriminative features and practical performance. That is, 2D vanilla convolution merges the feature maps generated by 3D octave convolutions along the channel direction, and homology shifting aggregates the information of the pixels locating at the same spatial position. Extensive experiments are conducted on four publicly available HSI datasets to evaluate the effectiveness and robustness of our model, and the results verify the superiority of Oct-MCNN-HS both in efficacy and efficiency.

The pistachio quality detection based on deep features plus unsupervised clustering

“…To address this issue, we mapped the feature vectors into 3D space, thus increasing the distance between data points of different categories and avoiding classification errors. While this increased the accuracy of the clustering, it also resulted in a longer clustering time due to the increased parameters, which require more calculation costs and thus slow down the training speed (Feng et al, 2021; Zhang et al, 2020). We hypothesized that the clustering performance would be better in a high‐dimensional space; however, due to the lack of data point visualization in a high‐dimensional space, it is difficult to manually evaluate the model's clustering performance.…”

“…To address this issue, we mapped the feature vectors into 3D space, thus increasing the distance between data points of different categories and avoiding classification errors. While this increased the accuracy of the clustering, it also resulted in a longer clustering time due to the increased parameters, which require more calculation costs and thus slow down the training speed (Feng et al, 2021;Zhang et al, 2020).…”

CMR-CNN: Cross-Mixing Residual Network for Hyperspectral Image Classification

“…To prove the effectiveness of our network, we select the following methods for comparing, including SVM [7], 2D-CNN [22], 3D-CNN [30], SSRN [38], DPRN [39], HybridSN [36], and the recently proposed methods OCT-MCNN [35], SAC-NET [24], MCNN-CP [34]. Meanwhile, for fairly comparing the performance of each method, we select OA, AA, and kappa coefficients as the evaluation criteria in experiments.…”

“…Covariance pooling techniques are used to extract second-order information from the spectralspatial feature maps, and channel shifts and weighting are used to highlight the importance of different spectral bands. Besides, Feng et al [35] proposed a hybrid convolutional neural network (OCT-MCNN) using 3D Octave and 2D Vanilla for HSI classification. In brief, the authors first utilized the spectral 3D convolution and the spatial 2D convolution to obtain hybrid feature maps, and then, employed covariance pooling to extract second-order information from the spectralspatial feature maps for HSI classification.…”

Hyperspectral Image Classification Network Based on 3D Octave Convolution and Multiscale Depthwise Separable Convolution

“…Octave convolution was introduced in [41] to lessen the geographically redundant feature information and lower the complexity of the models, as there is a significant degree of redundancy in the spatial information of the feature maps produced by CNNs. The feature fusion of 3D Octave convolution and 2D raw convolution in [42] improves the model classification accuracy and operation efficiency. A hyperspectral image classification method combining 3D Octave convolution and the bi-directional recurrent neural network attention network is proposed in the literature [43].…”