Remote Sensing Image Fusion with Convolutional Neural Network

Zhong, Jinying; Yang, Bin; Guoyu, Huang; Zhong, Fei; Chen, Zhongze

doi:10.1007/s11220-016-0135-6

Cited by 121 publications

(49 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A first attempt in this direction can be found in [120], where authors use a shallow network to upsample the intensity component obtained after the IHS of color images (RGB). Once the multispectral bands have been upsampled with the CNN, a traditional Gram-Schmidt transform is used to perform the pansharpening.…”

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,432

1,204

View full text Add to dashboard Cite

This is the pre-acceptance version, to read the final version please go to IEEE Geoscience and Remote Sensing Magazine on IEEE XPlore.Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a "black-box" solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization. Following this wave of success and thanks to the increased availability of data and computational resources, the use of deep learning in remote sensing is finally taking off in remote sensing as well. Remote sensing data bring some new challenges for deep learning, since satellite image analysis raises some unique questions that translate into challenging new scientific questions:• Remote sensing data are often multi-modal, e.g. from optical (multi-and hyperspectral) and synthetic aperture radar (SAR) sensors, where both the imaging geometries and the content are completely different. Data and information fusion uses these complementary

show abstract

Section: Multimodal Data Fusionmentioning

confidence: 99%

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Zhu

Tuia

Mou

et al. 2017

IEEE Geosci. Remote Sens. Mag.

2,432

1,204

View full text Add to dashboard Cite

show abstract

“…2 loss has been widely used while performing low level vision tasks [3,[6][7][8]. However, in this work we demonstrate that using 1 loss will generate better results than 2 on pansharpening task.…”

Section: Impact Of Loss Functionsmentioning

confidence: 67%

“…2. Instead of performing pan-sharpening in pixel level [6][7][8], we accomplish fusion in feature domain, which will reduce spectral distortion. This is because PAN and MS images contain different information.…”

Section: Two-stream Generatormentioning

confidence: 99%

See 1 more Smart Citation

Psgan: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

Liu

Wang

Liu

2018

2018 25th IEEE International Conference on Image Processing (ICIP)

102

View full text Add to dashboard Cite

Remote sensing image fusion (also known as pan-sharpening) aims to generate a high resolution multi-spectral image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral (MS) image. In this paper, we propose PSGAN, a generative adversarial network (GAN) for remote sensing image pansharpening. To the best of our knowledge, this is the first attempt at producing high quality pan-sharpened images with GANs. The PSGAN consists of two parts. Firstly, a two-stream fusion architecture is designed to generate the desired high resolution multi-spectral images, then a fully convolutional network serving as a discriminator is applied to distinct "real" or "pan-sharpened" MS images. Experiments on images acquired by Quickbird and GaoFen-1 satellites demonstrate that the proposed PSGAN can fuse PAN and MS images effectively and significantly improve the results over the state of the art traditional and CNN based pan-sharpening methods.

show abstract

“…They utilized a three-layered CNN architecture, which was originally designed for image super-resolution [19], to achieve pan-sharpening. Zhong et al [24] presented a CNN based hybrid pan-sharpening method, in which CNN was employed to enhance the spatial resolution of the MS image, then the GS transform was utilized to fuse the enhanced MS and PAN image to obtain the pan-sharpened images. The network used to enhance the spatial resolution of MS image is also a three-layered CNN similar to SRCNN [19].…”

Section: Introductionmentioning

confidence: 99%

Remote sensing image fusion based on two-stream fusion network

2020

View full text Add to dashboard Cite

Remote sensing image fusion (also known as pan-sharpening) aims at generating high resolution multi-spectral (MS) image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral image. Inspired by the astounding achievements of convolutional neural networks (CNNs) in a variety of computer vision tasks, in this paper, we propose a two-stream fusion network (TFNet) to address the problem of pan-sharpening. Unlike previous CNN based methods that consider pan-sharpening as a super resolution problem and perform pan-sharpening in pixel level, the proposed TFNet aims to fuse PAN and MS images in feature level and reconstruct the pan-sharpened image from the fused features. The TFNet mainly consists of three parts. The first part is comprised of two networks extracting features from PAN and MS images, respectively. The subsequent network fuses them together to form compact features that represent both spatial and spectral information of PAN and MS images, simultaneously. Finally, the desired high spatial resolution MS image is recovered from the fused features through an image reconstruction network. Experiments on Quickbird and GaoFen-1 satellite images demonstrate that the proposed TFNet can fuse PAN and MS images, effectively, and produce pan-sharpened images competitive with even superior to state of the arts.

show abstract

Remote Sensing Image Fusion with Convolutional Neural Network

Cited by 121 publications

References 19 publications

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Deep Learning in Remote Sensing: A Comprehensive Review and List of Resources

Psgan: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

Remote sensing image fusion based on two-stream fusion network

Contact Info

Product

Resources

About