Deep Manifold Embedding for Hyperspectral Image Classification

Gong, Zhiqiang; Hu, Weidong; Du, Xiaoyong; Zhong, Ping; Hu, Panhe

doi:10.1109/tcyb.2021.3069790

Cited by 25 publications

(6 citation statements)

References 75 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is noteworthy that like many other data analysis techniques such as PCA, FEM, GSE 50 , CCSF 39 , t-SNE, and UMAP, interactive relationships of the components inside an HD feature point are not considered explicitly during the MDA embedding process 51 . For some special applications such as the assessment of image similarity in a low dimensional embedding, DNN-based manifold learning techniques like Deep Manifold Embedding Method (DMEM) 52 and Deep Local-flatness Manifold Embedding (DLME) 53 could be used. In this process, however, MDA is also useful as it offers an effective method for analyzing the deep learning features and sheds insights into the embedding process.…”

Section: Discussionmentioning

confidence: 99%

Revealing hidden patterns in deep neural network feature space continuum via manifold learning

Islam,

Zhou,

Ren

et al. 2023

Nat Commun

View full text Add to dashboard Cite

Deep neural networks (DNNs) extract thousands to millions of task-specific features during model training for inference and decision-making. While visualizing these features is critical for comprehending the learning process and improving the performance of the DNNs, existing visualization techniques work only for classification tasks. For regressions, the feature points lie on a high dimensional continuum having an inherently complex shape, making a meaningful visualization of the features intractable. Given that the majority of deep learning applications are regression-oriented, developing a conceptual framework and computational method to reliably visualize the regression features is of great significance. Here, we introduce a manifold discovery and analysis (MDA) method for DNN feature visualization, which involves learning the manifold topology associated with the output and target labels of a DNN. MDA leverages the acquired topological information to preserve the local geometry of the feature space manifold and provides insightful visualizations of the DNN features, highlighting the appropriateness, generalizability, and adversarial robustness of a DNN. The performance and advantages of the MDA approach compared to the existing methods are demonstrated in different deep learning applications.

show abstract

Section: Discussionmentioning

confidence: 99%

Revealing hidden patterns in deep neural network feature space continuum via manifold learning

Islam,

Zhou,

Ren

et al. 2023

Nat Commun

View full text Add to dashboard Cite

show abstract

“…This approach harnesses the spectral similarities and differences between samples in the dataset to improve the performance of deep models. Furthermore, more advanced avenue of research explores the incorporation of the physical properties inherent to categories within hyperspectral data for the construction of training loss functions, such as Statistical loss [30], DMEM loss [31]. By considering the unique spectral characteristics and physical attributes of materials or objects of the same category, it becomes possible to design loss functions that promote a deeper understanding and better exploitation of these intrinsic properties.…”

Section: Introductionmentioning

confidence: 99%

Adversarial system of one-pixel attack for hyperspectral image classification

Peng,

Zhou,

Sun

et al. 2023

Preprint

View full text Add to dashboard Cite

The introduction of deep learning arouses great success in hyperspectral image (HSI) classification. However, there is a growing concern about adversarial attacks on deep models, which can significantly impact the performance of HSI classification through imperceptible perturbations. In this paper, we first introduce a semi-black-box attack method to hyperspectral domain, one-pixel attack, which fools the HSI classifier by modifying only one pixel with high intensity. It is verified on standard datasets and a well-known HSI classification model, HybridSN. An adversarial system based on one-pixel attack (OPAS) is then proposed by integrating adversarial training into model training. The system not only improves the robustness of the classification model but also enhances the model’s ability to defend against other adversarial attacks. Experimental results demonstrate show that OPAS can achieve nearly 100% defense effectiveness against some basic adversarial attacks. Additionally, an HSI adversarial map is created to visualize one-pixel attack. Through it can be observed the characteristic that one-pixel attack can attack classification boundaries, and this property is further verified in hyperspectral field.

show abstract

“…H YPERSPECTRAL images (HSIs) have been widely studied and applied in different tasks with the aim of processing and analyzing their sheer amount of information. This has been done through various techniques, such as image classification [1]- [5], data fusion [6]- [8], target detection [9]- [11], anomaly detection [12]- [15], data denoising [16]- [18], and so on. In this context, hyperspectral unmixing [19]- [23] is one of the most important applications of HSI data processing.…”

Section: Introductionmentioning

confidence: 99%

A New Deep Convolutional Network for Effective Hyperspectral Unmixing

Tao

Paoletti

Han

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Hyperspectral unmixing extracts pure spectral constituents (endmembers) and their corresponding abundance fractions from remotely sensed scenes. Most traditional hyperspectral unmixing methods require the results of other endmember extraction algorithms to complete the abundance estimation step. Due to the impressive learning and data fitting capabilities of convolutional neural networks (CNNs), deep learning (DL)-based hyperspectral unmixing technologies have rapidly developed in the literature. According to the procedure used to combine different layers (i.e., fully connected layers, convolution layers, and activation layers), these techniques are mainly divided into three main categories, i.e., those based on autoencoder networks, convolutional neural networks, and convolutional autoencoder networks. They usually extract the weight and output of a specific activation layer as endmember signatures and abundance maps, respectively. Moreover, most existing DL-based unmixing approaches usually use 2D CNNs to learn the features contained in hyperspectral images, and very few approaches employ 3D CNNs to extract spectral and spatial information. However, 2D CNN-based techniques cannot capture good discriminative feature maps from the spectral viewpoint, and 3D CNN-based techniques usually have high computational overload. In this work, to further exploit the feature extraction capability of CNNs, we combine 3D and 2D convolutions to propose a crossconvolution unmixing network (CrossCUN) for hyperspectral unmixing. Simultaneously, to better illustrate the improvements of our proposed CrossCUN, we also build the corresponding 2D convolution unmixing network (2DCUN) and 3D convolution unmixing network (3DCUN). We evaluate the performance of our newly developed networks on two types of synthetic data sets and three real hyperspectral images. Experimental results show that the proposed networks not only obtain better results than other DL-based unmixing methods but also do not require any prior knowledge (e.g., the results of other endmember extraction

show abstract

Deep Manifold Embedding for Hyperspectral Image Classification

Cited by 25 publications

References 75 publications

Revealing hidden patterns in deep neural network feature space continuum via manifold learning

Revealing hidden patterns in deep neural network feature space continuum via manifold learning

Adversarial system of one-pixel attack for hyperspectral image classification

A New Deep Convolutional Network for Effective Hyperspectral Unmixing

Contact Info

Product

Resources

About