Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

Jiang, Xuefeng; Liu, Wenbo; Zhang, Yue; Liu, Junrui; Li, Shuying; Lin, Jianzhe

doi:10.1109/lgrs.2020.2991405

Cited by 22 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remote Sens. 2022, 14, x FOR PEER REVIEW 2 of 31 vision and pattern recognition [6,7], the classification methods based on Convolutional Neural Network (CNN) [8][9][10][11][12][13][14][15][16][17][18][19] have been widely investigated for they can learn and extract image features automatically. Hu et al [9] and Du et al [10] used a pretrained CNN to extract image features.…”

Section: The Dilemma Of Existing Rsisc Methodsmentioning

confidence: 99%

“…One is the traditional machine learning-based methods with hand-crafted features, such as models based on Bag of Visual Words (BoVW) [2], Randomized Spatial Partition (RSP) [3], Hierarchical Coding Vector (HCV) [4] and Fisher vectors (FVs) [5]. As deep learning technology has been proved to have excellent performance in computer vision and pattern recognition [6,7], the classification methods based on Convolutional Neural Network (CNN) [8][9][10][11][12][13][14][15][16][17][18][19] have been widely investigated for they can learn and extract image features automatically. Hu et al [9] and Du et al [10] used a pretrained CNN to extract image features.…”

Section: Introductionmentioning

confidence: 99%

“…Wang et al [13] adopted an attention module and constructed an end-to-end network. Li et al [14], Jiang et al [15], and Zhang et al [16] used Capsule Network (CapsNet) in their model which can preserve the spatial information of RS images. To date, the RSISC tasks have achieved a quite high-level accuracy on standard datasets.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

Zhang

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

At present, the classification accuracy of high-resolution Remote Sensing Image Scene Classification (RSISC) has reached a quite high level on standard datasets. However, when coming to practical application, the intrinsic noise of satellite sensors and the disturbance of atmospheric environment often degrade real Remote Sensing (RS) images. It introduces defects to them, which affects the performance and reduces the robustness of RSISC methods. Moreover, due to the restriction of memory and power consumption, the methods also need a small number of parameters and fast computing speed to be implemented on small portable systems such as unmanned aerial vehicles. In this paper, a Lightweight Progressive Inpainting Network (LPIN) and a novel combined approach of LPIN and the existing RSISC methods are proposed to improve the robustness of RSISC tasks and satisfy the requirement of methods on portable systems. The defects in real RS images are inpainted by LPIN to provide a purified input for classification. With the combined approach, the classification accuracy on RS images with defects can be improved to the original level of those without defects. The LPIN is designed on the consideration of lightweight model. Measures are adopted to ensure a high gradient transmission efficiency while reducing the number of network parameters. Multiple loss functions are used to get reasonable and realistic inpainting results. Extensive tests of image inpainting of LPIN and classification tests with the combined approach on NWPU-RESISC45, UC Merced Land-Use and AID datasets are carried out which indicate that the LPIN achieves a state-of-the-art inpainting quality with less parameters and a faster inpainting speed. Furthermore, the combined approach keeps the comparable classification accuracy level on RS images with defects as that without defects, which will improve the robustness of high-resolution RSISC tasks.

show abstract

Section: The Dilemma Of Existing Rsisc Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

Zhang

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Another model framework, the CapsNet proposed by Professor Hinton [21], which not only greatly reduces the size of mode, but also makes more effective use of spatial location information, and it better encodes the relationship between local information and global goal. In some studies, it has been gradually confirmed to have better performance on classification tasks, which are of a limited number and low resolution datasets [22][23][24][25][26] compared to some CNN models, such as AlexNet [27], NDCNN [28], and NPMIL [29].…”

Section: Introductionmentioning

confidence: 99%

TongueCaps: An Improved Capsule Network Model for Multi-Classification of Tongue Color

Yan

Jiang

2022

Diagnostics

View full text Add to dashboard Cite

Tongue color is an important part of tongue diagnosis. The change of tongue color is affected by pathological state of body, blood rheology, and other factors. Therefore, physicians can understand a patient’s condition by observing tongue color. Currently, most studies use machine learning, which is time consuming and labor intensive. Other studies use deep learning based on convolutional neural network (CNN), but the affine transformation of CNN is less robust and easily loses the spatial relationship between features. Recently, Capsule Networks (CapsNet) have been proposed to overcome these problems. In our work, CapsNet is used for tongue color research for the first time, and improved model TongueCaps is proposed, which combines the advantage of CapsNet and residual block structure to achieve end to end tongue color classification. We conduct experiments on 1371 tongue images; TongueCaps achieve accuracy is 0.8456, sensitivity is 0.8474, and specificity is 0.9586. In addition, the size of TongueCaps is 8.11 M, and FLOPs is 1,335,342, which are smaller than CNN in comparison models. Experiments have confirmed that the CapsNet can be used for tongue color research, and improved model TongueCaps, in this paper, is superior to other comparison models in terms of accuracy, specificity and sensitivity, computational complexity, and size of model.

show abstract

“…Li et al [43] introduced a dual channel CapsNet to fuse the hyperspectral data and LiDAR-derived elevation data and address the noise problem for HSI classification. Jiang et al [44] proposed a dual-channel capsule network which can extract the features from spectral and spatial domains with two separate channels for HSI classification. Ding et al [45] designed an adaptive CapsNet composed of an adaptive routing algorithm and the powered activation regularization method for HIS classification which can amplify the gradient and learn the sparser and more discriminative representation.…”

Section: Introductionmentioning

confidence: 99%

Best Representation Branch Model for Remote Sensing Image Scene Classification

Zhang

Sun³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Remote sensing image scene classification is an important method for understanding the high-resolution remote sensing images. Based on Convolutional Neural Network, various classification methods have been applied into this field and achieved remarkable results. These methods mainly rely on the semantic information to improve the classification performance. However, as the network goes deeper, the highly abstract and global semantic information makes it difficult for the network to accurately classify scene images with similar layout and structures, limiting the further improvement of classification accuracy. Relying on the semantic information only is not sufficient to effectively classify these similar scene images and the network needs spatial information to enhance the classification capability. To solve this dilemma, this paper proposes a best representation branch model, which reaches the optimal balance point where the network can make use of both the semantic information and spatial information to improve the final classification accuracy. In the proposed method, ResNet50 pretrained on the ImageNet dataset is first divided into four branches with different depths to extract feature maps and Capsule Network is used as the classifier. The Grad-CAM algorithm is adopted to explain the mechanism of the optimal balance point from the perspective of attention and guide the further feature fusion. In addition, ablation studies are conducted to prove the effectiveness of our method and extensive experiments are conducted on three public benchmark remote sensing datasets. The results demonstrate that the proposed method can achieve competitive classification performance compared to the state-ofthe-art methods.

show abstract

Spectral–Spatial Hyperspectral Image Classification Using Dual-Channel Capsule Networks

Cited by 22 publications

References 18 publications

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

TongueCaps: An Improved Capsule Network Model for Multi-Classification of Tongue Color

Best Representation Branch Model for Remote Sensing Image Scene Classification

Contact Info

Product

Resources

About