Hyperspectral image super-resolution with spectral–spatial network

Jia, Jinrang; Ji, Liu; Zhao, Yongchao; Geng, Xiurui

doi:10.1080/01431161.2018.1471546

Cited by 29 publications

(15 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Within the scope of this work the term end-to-end refers to network architectures that take the HSI volume as input and produce the target data without using separate pre- or post- processing stages. Other approaches are composed of multiple stages in which CNNs are applied extensively as in [ 87 , 88 ] or, more interestingly, without requiring auxiliary images, as in [ 89 ].…”

Section: Hsi Applications Meet DL Solutionsmentioning

confidence: 99%

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

et al. 2019

View full text Add to dashboard Cite

Modern hyperspectral imaging systems produce huge datasets potentially conveying a great abundance of information; such a resource, however, poses many challenges in the analysis and interpretation of these data. Deep learning approaches certainly offer a great variety of opportunities for solving classical imaging tasks and also for approaching new stimulating problems in the spatial–spectral domain. This is fundamental in the driving sector of Remote Sensing where hyperspectral technology was born and has mostly developed, but it is perhaps even more true in the multitude of current and evolving application sectors that involve these imaging technologies. The present review develops on two fronts: on the one hand, it is aimed at domain professionals who want to have an updated overview on how hyperspectral acquisition techniques can combine with deep learning architectures to solve specific tasks in different application fields. On the other hand, we want to target the machine learning and computer vision experts by giving them a picture of how deep learning technologies are applied to hyperspectral data from a multidisciplinary perspective. The presence of these two viewpoints and the inclusion of application fields other than Remote Sensing are the original contributions of this review, which also highlights some potentialities and critical issues related to the observed development trends.

show abstract

Section: Hsi Applications Meet DL Solutionsmentioning

confidence: 99%

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Jia et. al [44] combined a spatial network and a spectral network serially to take full use of spatial and spectral information. To address the spectral disorder caused by 2D convolution, Zheng et al [45] proposed a separable-spectral and inception network (SSIN) to enhance the resolution in a coarse-to-fine manner.…”

Section: Related Workmentioning

confidence: 99%

Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

Cui

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Hyperspectral image (HSI) super-resolution (SR) is of great application value and has attracted broad attention. The hyperspectral single image super-resolution (HSISR) task is correspondingly difficult in SR due to the unavailability of auxiliary high resolution images. To tackle this challenging task, different from the existing learning-based HSISR algorithms, in this paper we propose a novel framework, i.e., a 1D-2D attentional convolutional neural network, which employs a separation strategy to extract the spatial-spectral information and then fuse them gradually. More specifically, our network consists of two streams: a spatial one and a spectral one. The spectral one is mainly composed of the 1D convolution to encode a small change in the spectrum, while the 2D convolution, cooperating with the attention mechanism, is used in the spatial pathway to encode spatial information. Furthermore, a novel hierarchical side connection strategy is proposed for effectively fusing spectral and spatial information. Compared with the typical 3D convolutional neural network (CNN), the 1D-2D CNN is easier to train with less parameters. More importantly, our proposed framework can not only present a perfect solution for the HSISR problem, but also explore the potential in hyperspectral pansharpening. The experiments over widely used benchmarks on SISR and hyperspectral pansharpening demonstrate that the proposed method could outperform other state-of-the-art methods, both in visual quality and quantity measurements. a feasible scheme is to increase the pixel spatial size. However, because of fundamental physical limits in practice, it is difficult to improve the sensor capability. Consequently, compared with conventional RGB or multispectral cameras, the obtained hyperspectral image (HSI) is always with a relative low spatial resolution, which limits their practical applications. In recent years, HSI super-resolution (SR) has attracted more and more attention in the remote sensing community.HSI SR is a promising signal post-processing technique aiming at acquiring a high resolution (HR) image from its low resolution (LR) version to overcome the inherent resolution limitations [1]. Generally, we can roughly divide this technique into two categories, according to the availability of an HR auxiliary image, e.g., the single HSI super-resolution or pan sharpening methods with the HR panchromatic image. For example, the popular single HSI super-resolution methods are bilinear [2] and bicubic interpolation [3] based on interpolation, and [4,5] based on regularization. Pansharpening methods can be roughly divided into five categories: component substitution (CS) [6][7][8], which may cause spectral distortion; multiresolution analysis (MRA) [9][10][11][12], which can keep spectral consistency at the cost of much computation and great complexity of parameter setting; bayesian methods [13][14][15] and matrix factorization [16], which can achieve prior spatial and spectral performance at a very high computational cost; and hybrid metho...

show abstract

“…However, due to the limitation of sensors, the spatial resolution of HSIs is relatively low. It is very important to improve the spatial resolution of HSIs [3][4][5].…”

Section: Introductionmentioning

confidence: 99%

“…We use MSC to fuse the feature maps with different scales of adjacent groups. For instance,3 is obtained by convolving the sum of the two paths. One path consists of 1 convolutional layer and the effective receptive fields of 2 are 3×3×3.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Hyperspectral Image Super Resolution Based on Multiscale Feature Fusion and Aggregation Network With 3-D Convolution

Tang

2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The spectral resolution of hyperspectral images (HSIs) is very high. Nevertheless, their spatial resolution is low due to various hardware limitations. Therefore, it is important to study HSI super-resolution to improve their spatial resolution. In this paper, for hyperspectral single-image super-resolution, we propose a multi-scale feature fusion and aggregation network with 3D convolution (MFFA-3D) by cascading the MFFA-3D block. The MFFA-3D block includes group multi-scale feature fusion part and multi-scale feature aggregation part. In group multi-scale feature fusion part, a novel group multi-scale feature fusion method is proposed. Group feature fusion module and two-step multi-scale module are proposed in multi-scale feature aggregation part. In order to prevent spectral distortion, a spectral gradient loss function is proposed and combined with the mean square error (MSE) loss function to form the final loss function. Since the proposed super-resolution network is full 3D convolutional network, our method can perform direct super-resolution transfer even if the number of the bands of test images is different from that of training images. The experiments over simulated and real hyperspectral images demonstrate the superiority of the proposed method in terms of qualitative and quantitative evaluation.

show abstract

Hyperspectral image super-resolution with spectral–spatial network

Cited by 29 publications

References 33 publications

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

Deep Learning Meets Hyperspectral Image Analysis: A Multidisciplinary Review

Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

Hyperspectral Image Super Resolution Based on Multiscale Feature Fusion and Aggregation Network With 3-D Convolution

Contact Info

Product

Resources

About