Hyperspectral Image Superresolution by Transfer Learning

Yuan, Yuan; Zheng, Xiangtao; Lu, Xiaoqiang

doi:10.1109/jstars.2017.2655112

Cited by 225 publications

(124 citation statements)

References 43 publications

Supporting

Mentioning

124

Contrasting

Order By: Relevance

“…In [122] authors use a super-resolution CNN trained on natural images [123] as a pre-trained model and fine-tune it on a dataset of hyperspectral images. By doing so, they make an attempt at transfer leaning [124] between the domains of color (three bands, large bandwidths) and hyperspectral images (many bands, narrow bandwidths).…”

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,439

1,204

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version please go to IEEE Geoscience and Remote Sensing Magazine on IEEE XPlore.Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a "black-box" solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization. Following this wave of success and thanks to the increased availability of data and computational resources, the use of deep learning in remote sensing is finally taking off in remote sensing as well. Remote sensing data bring some new challenges for deep learning, since satellite image analysis raises some unique questions that translate into challenging new scientific questions:• Remote sensing data are often multi-modal, e.g. from optical (multi-and hyperspectral) and synthetic aperture radar (SAR) sensors, where both the imaging geometries and the content are completely different. Data and information fusion uses these complementary

show abstract

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,439

1,204

View full text Add to dashboard Cite

show abstract

“…Recently, transfer learning [37,38] has been employed to solve the task of hyperspectral super-resolution and pansharpening. However, the spectral distortions may not be solved well by transferring directly from the network used in RGB image super-resolution methods to hyperspectral image super-resolution [39]. Mei et al [40] proposed a 3D CNN architecture to encode spatial and spectral information jointly.…”

Section: Related Workmentioning

confidence: 99%

Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

Cui

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Hyperspectral image (HSI) super-resolution (SR) is of great application value and has attracted broad attention. The hyperspectral single image super-resolution (HSISR) task is correspondingly difficult in SR due to the unavailability of auxiliary high resolution images. To tackle this challenging task, different from the existing learning-based HSISR algorithms, in this paper we propose a novel framework, i.e., a 1D-2D attentional convolutional neural network, which employs a separation strategy to extract the spatial-spectral information and then fuse them gradually. More specifically, our network consists of two streams: a spatial one and a spectral one. The spectral one is mainly composed of the 1D convolution to encode a small change in the spectrum, while the 2D convolution, cooperating with the attention mechanism, is used in the spatial pathway to encode spatial information. Furthermore, a novel hierarchical side connection strategy is proposed for effectively fusing spectral and spatial information. Compared with the typical 3D convolutional neural network (CNN), the 1D-2D CNN is easier to train with less parameters. More importantly, our proposed framework can not only present a perfect solution for the HSISR problem, but also explore the potential in hyperspectral pansharpening. The experiments over widely used benchmarks on SISR and hyperspectral pansharpening demonstrate that the proposed method could outperform other state-of-the-art methods, both in visual quality and quantity measurements. a feasible scheme is to increase the pixel spatial size. However, because of fundamental physical limits in practice, it is difficult to improve the sensor capability. Consequently, compared with conventional RGB or multispectral cameras, the obtained hyperspectral image (HSI) is always with a relative low spatial resolution, which limits their practical applications. In recent years, HSI super-resolution (SR) has attracted more and more attention in the remote sensing community.HSI SR is a promising signal post-processing technique aiming at acquiring a high resolution (HR) image from its low resolution (LR) version to overcome the inherent resolution limitations [1]. Generally, we can roughly divide this technique into two categories, according to the availability of an HR auxiliary image, e.g., the single HSI super-resolution or pan sharpening methods with the HR panchromatic image. For example, the popular single HSI super-resolution methods are bilinear [2] and bicubic interpolation [3] based on interpolation, and [4,5] based on regularization. Pansharpening methods can be roughly divided into five categories: component substitution (CS) [6][7][8], which may cause spectral distortion; multiresolution analysis (MRA) [9][10][11][12], which can keep spectral consistency at the cost of much computation and great complexity of parameter setting; bayesian methods [13][14][15] and matrix factorization [16], which can achieve prior spatial and spectral performance at a very high computational cost; and hybrid metho...

show abstract

“…Wan et al perform a comprehensive study on deep learning for CBIR with the goal of addressing the fundamental problem of feature representation in CBIR, bridging the semantic gap between the low-level image pixels captured by machines and the high-level semantic concepts perceived by human [33]. Inspired by the great success of deep learning, the remote sensing community has realized the potential of applying deep learning techniques for remote sensing tasks such as scene classification [34][35][36][37][38], object detection [39][40][41], semantic segmentation [42,43], image super-resolution [44] and image retrieval [45][46][47][48][49]. For RSIR, the deep learning techniques can be roughly divided into unsupervised feature learning and supervised feature learning methods.…”

Section: Introductionmentioning

confidence: 99%

PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval

Zhou

Newsam

et al. 2018

ISPRS Journal of Photogrammetry and Remote Sensing

398

223

View full text Add to dashboard Cite

Remote sensing image retrieval (RSIR), which aims to efficiently retrieve data of interest from large collections of remote sensing data, is a fundamental task in remote sensing.Over the past several decades, there has been significant effort to extract powerful feature representations for this task since the retrieval performance depends on the representative strength of the features. Benchmark datasets are also critical for developing, evaluating, and comparing RSIR approaches. Current benchmark datasets are deficient in that 1) they were originally collected for land use/land cover classification and not image retrieval; 2) they are relatively small in terms of the number of classes as well the number of sample images per class; and 3) the retrieval performance has saturated. These limitations have severely restricted the development of novel feature representations for RSIR, particularly the recent deep-learning based features which require large amounts of training data. We therefore present in this paper, a new large-scale remote sensing dataset termed "PatternNet" that was collected specifically for RSIR. PatternNet was collected from high-resolution imagery and contains 38 classes with 800 images per class. We also provide a thorough review of RSIR approaches ranging from traditional handcrafted feature based methods to recent deep learning based ones. We evaluate over 35 methods to establish extensive baseline results for future RSIR research using the PatternNet benchmark Keywords: remote sensing, content based image retrieval (CBIR), benchmark dataset, handcrafted features, deep learning, convolutional neural networks

show abstract

Hyperspectral Image Superresolution by Transfer Learning

Cited by 225 publications

References 43 publications

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Hyperspectral Image Super-Resolution with 1D–2D Attentional Convolutional Neural Network

PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval

Contact Info

Product

Resources

About