Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms

Abstract:Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.

show abstract

“…. , 2 19 , 2 20 ]. The parameter involved in the CCF classifier is the number of trees; we set it to 200 in the experiment.…”

Section: Applications On Real Data Fusionmentioning

confidence: 99%

“…Several HSI-MSI fusion algorithms have been proposed in the last decades [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. HR HSI can be reconstructed by combining endmember of LR HSI and an abundance of HR MSI.…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

2018

View full text Add to dashboard Cite

show abstract

“…The second component utilizes the HCM algorithm that fuses a highresolution color image with an enhanced hyperspectral image coming out of the first component. Recently, HCM has been applied to several applications, including enhancing Worldview-3 images [49], fusion of Landsat and MODIS images [50], pansharpening of Mastcam images [51], and fusing of THEMIS and TES [52].…”

Section: Incorporation Of Psf Into Hcmmentioning

confidence: 99%

A New Pansharpening Approach for Hyperspectral Images

Kwan¹,

Zhou²,

Budavari³

2018

Colorimetry and Image Processing

Self Cite

View full text Add to dashboard Cite

show abstract

“…Worldview images have a super high resolution of 0.5 m for pansharpened images 1 . However, the revisit times are rare.…”

Section: Introductionmentioning

confidence: 99%

Fusion of landsat and worldview images

Kwan¹,

Chou

Yang

et al. 2019

Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII

Self Cite

View full text Add to dashboard Cite

Pansharpened Landsat images have 15 m spatial resolution with 16-day revisit periods. On the other hand, Worldview images have 0.5 m resolution after pansharpening but the revisit times are uncertain. We present some preliminary results for a challenging image fusion problem that fuses Landsat and Worldview (WV) images to yield a high temporal resolution image sequence at the same spatial resolution of WV images. Since the spatial resolution between Landsat and Worldview is 30 to 1, our preliminary results are mixed in that the objective performance metrics such as peak signal-to-noise ratio (PSNR), correlation coefficient (CC), etc. sometimes showed good fusion performance, but at other times showed poor results. This indicates that more fusion research is still needed in this niche application.

show abstract

Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms

Cited by 99 publications

References 16 publications

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

A New Pansharpening Approach for Hyperspectral Images

Fusion of landsat and worldview images

Contact Info

Product

Resources

About