Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature

Yokoya, Naoto; Grohnfeldt, Claas; Chanussot, Jocelyn

doi:10.1109/mgrs.2016.2637824

Cited by 576 publications

(446 citation statements)

References 71 publications

Supporting

Mentioning

441

Contrasting

Unclassified

Order By: Relevance

“…In the results presented below, "ME" is the percent Mean Error, (predicted spectra-observ spectra)/observed spectra; "MAE" is the percent Mean Absolute Error; "RMSE" is the Root Mean Squ of Error; and "STD_AE" is the Standard Deviation of the percent Absolute Error. For measures used hyperspectral image quality evaluation [31,32], "SNR" is the signal to noise ratio; "UIQI" is the univer Each spectrum is given a random color.…”

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

“…In the results presented below, "ME" is the percent Mean Error, (predicted spectra-observed spectra)/observed spectra; "MAE" is the percent Mean Absolute Error; "RMSE" is the Root Mean Square of Error; and "STD_AE" is the Standard Deviation of the percent Absolute Error. For measures used in hyperspectral image quality evaluation [31,32], "SNR" is the signal to noise ratio; "UIQI" is the universal image quality index; "SAM" is spectral angle mapper; "ERGAS" is the relative dimensionless global error in synthesis, which calculates the amount of spectral distortion in the image; and "DD" is the degree of distortion. With the exception of SNR and UIQI, smaller values are better for all measures.…”

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

“…* Metrics for the spline approach were calculated in the wavelength range that is valid for the multispectral image (490-800 nm) rather than the full 400-800 nm range. The accuracy metrics are defined in the text: "ME" is the percent Mean Error, (predicted spectra-observed spectra)/observed spectra; "MAE" is the percent Mean Absolute Error; "RMSE" is the Root Mean Square of Error; and "STD_AE" is the Standard Deviation of the percent Absolute Error, "SNR" is the signal to noise ratio; "UIQI" is the universal image quality index; "SAM" is spectral angle mapper; "ERGAS" is the relative dimensionless global error in synthesis, which calculates the amount of spectral distortion in the image; and "DD" is the degree of distortion [31,32]. Bold text indicates best performance in a column.…”

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

“…Apart from these three categories of hyperspectral prediction approaches, there have been many studies on the data fusion of hyperspectral imagery and multispectral imagery (e.g., [31,32]). Hyperspectral-multispectral (HS-MS) image fusion, however, is a different problem from the hyperspectral prediction studied here.…”

Section: Study Sitementioning

confidence: 99%

See 3 more Smart Citations

Fusion of Multispectral Imagery and Spectrometer Data in UAV Remote Sensing

et al. 2017

View full text Add to dashboard Cite

High spatial resolution hyperspectral data often used in precision farming applications are not available from current satellite sensors, and difficult or expensive to acquire from standard aircraft. Alternatively, in precision farming, unmanned aerial vehicles (UAVs) are emerging as lower cost and more flexible means to acquire very high resolution imagery. Miniaturized hyperspectral sensors have been developed for UAVs, but the sensors, associated hardware, and data processing software are still cost prohibitive for use by individual farmers or small remote sensing firms. This study simulated hyperspectral image data by fusing multispectral camera imagery and spectrometer data. We mounted a multispectral camera and spectrometer, both being low cost and low weight, on a standard UAV and developed procedures for their precise data alignment, followed by fusion of the spectrometer data with the image data to produce estimated spectra for all the multispectral camera image pixels. To align the data collected from the two sensors in both the time and space domains, a post-acquisition correlation-based global optimization method was used. Data fusion, to estimate hyperspectral reflectance, was implemented using several methods for comparison. Flight data from two crop sites, one being tomatoes, and the other corn and soybeans, were used to evaluate the alignment procedure and the data fusion results. The data alignment procedure resulted in a peak R 2 between the spectrometer and camera data of 0.95 and 0.72, respectively, for the two test sites. The corresponding multispectral camera data for these space and time offsets were taken as the best match to a given spectrometer reading, and used in modelling to estimate hyperspectral imagery from the multispectral camera pixel data. Of the fusion approaches evaluated, principal component analysis (PCA) based models and Bayesian imputation reached a similar accuracy, and outperformed simple spline interpolation. Mean absolute error (MAE) between predicted and observed spectra was 17% relative to the mean of the observed spectra, and root mean squared error (RMSE) was 0.028. This approach to deriving estimated hyperspectral image data can be applied in a simple fashion at very low cost for crop assessment and monitoring within individual fields.

show abstract

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

Section: Comparison Of Data Fusion Methodsmentioning

confidence: 99%

Section: Study Sitementioning

confidence: 99%

See 2 more Smart Citations

Fusion of Multispectral Imagery and Spectrometer Data in UAV Remote Sensing

et al. 2017

View full text Add to dashboard Cite

show abstract

“…Compared with HSIs, multispectral images (MSI) have wider bandwidth, and are often of higher spatial resolution (e.g., ASTER MSI is of 15 m resolution). Fusing low resolution (LR) HSIs with a high resolution (HR) MSIs is an important technology to enhance the spatial resolution of HSI [4,5].…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

2018

View full text Add to dashboard Cite

Abstract:Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.

show abstract

Bibliography

2021

Deep Learning for the Earth Sciences

View full text Add to dashboard Cite

Hyperspectral and Multispectral Data Fusion: A comparative review of the recent literature

Abstract: International audienc

Cited by 576 publications

References 71 publications

Fusion of Multispectral Imagery and Spectrometer Data in UAV Remote Sensing

Fusion of Multispectral Imagery and Spectrometer Data in UAV Remote Sensing

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Bibliography

Contact Info

Product

Resources

About