Hyperspectral Unmixing Using Transformer Network

Abstract

Transformers have intrigued the vision research community with their state-of-the-art performance in natural language processing. With their superior performance, transformers have found their way in the field of hyperspectral image classification and achieved promising results. In this article, we harness the power of transformers to conquer the task of hyperspectral unmixing and propose a novel deep neural network-based unmixing model with transformers. A transformer network captures nonlocal feature dependencies by interactions between image patches, which are not employed in CNN models, and hereby has the ability to enhance the quality of the endmember spectra and the abundance maps. The proposed model is a combination of a convolutional autoencoder and a transformer. The hyperspectral data is encoded by the convolutional encoder. The transformer captures long-range dependencies between the representations derived from the encoder. The data are reconstructed using a convolutional decoder. We applied the proposed unmixing model to three widely used unmixing datasets, i.e., Samson, Apex, and Washington DC mall and compared it with the state-of-the-art in terms of root mean squared error and spectral angle distance. The source code for the proposed model will be made publicly available at https://github.com/preetam22n/DeepTrans-HSU.

DSET-Net: deep self-embedded transformer network for hyperspectral unmixing

“…We evaluate the proposed DSET-Net compared with several classical unmixing algorithms, such as the fully constrained least squares(FCLS) [16], the untied denoising autoencoder with sparsity (uDAS) [17], the deep autoencoder network (DAEN) [18], the cycle-consistency unmixing network (CyCU-Net) [19], the hyperspectral unmixing using deep image prior (UnDIP) [20], and the DeepTrans-HSU [10]. The quantitative results are shown in Table 1.…”

“…As a neural network mainly based on the self-attention mechanism, the transformer can better explore the potential relationships of different features, which improves the ability of the autoencoder to aggregate spatial and spectral correlations(SSCs) from pixels, thus improving abundance learning and the accuracy of unmixing [9]. However, the original transformer only uses the block sequences to obtain spectral information of pixels, ignoring the spatial correlation between pixels inside the blocks, causing negative effects to unmixing [10].…”

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

“…First. The MSA allows the free flow of information between tokens 45 , facilitating the exchange of information to focus on the interrelationships between plant, leaf disease and severity features 49 . It enhances the ability to identify plant disease characteristics of the multi-label identification model.…”

A Supervised Approach for Estimating Fractional Abundances of Binary Intimate Mixtures

“…This is generally done by utilizing the encoder, while the decoder transforms the estimated fractional abundances into reconstructed hyperspectral images using linear layers, where the endmembers act as the weights. A variety of autoencoders has been applied in hyperspectral unmixing: Denoising autoencoders [4], Sparse nonnegative autoencoders [5], Variational autoencoders [6], [7], Convolutional autoencoders [8], [9], [10], [11], Adversarial autoencoders [12], [13], and Transformer autoencoders ( [14], [15]).…”