Two-Path Network with Feedback Connections for Pan-Sharpening in Remote Sensing

Fu, Shipeng; Wang, Meng; Jeon, Gwanggil; Chehri, Abdellah; Zhang, Rongzhu; Yang, Xiaobo

doi:10.3390/rs12101674

Cited by 16 publications

(8 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In a subsequent work, Liu et al [43] proposed an improvement on TFNet, called ResTFNet, that further improves the performance of the proposed network by using basic residual blocks instead of the continuous convolutional layer in TFNet. Inspired by the dual-stream network structure, Fu et al [44] proposed a network structure called TPNwFB that, after extracting spatial and spectral information, introduces a feedback connectivity mechanism to implement a subnetwork iterative process using recurrent structures, which allows strong-deep feature backflow to modify poor low-level features.…”

Section: Cnn-based Pan-sharpeningmentioning

confidence: 99%

“…Li et al [50] carefully designed a feedback block to extract powerful high-level representations for low-level computer-vision tasks and transmit high-level representations to perfect low-level functions. Fu et al [44] added this feedback connection mechanism for super-resolution tasks to the network for pan-sharpening. They enable the featureextraction block to generate more powerful features by iterating the information in each subnetwork to the same module of the next subnetwork, iteratively up and downsampling the input features to achieve the feedback connectivity mechanism.…”

Section: Feedback Connection Structurementioning

confidence: 99%

“…One model-based method, PPXS [58], was considered. We selected five deep learning-based methods as contrast objects, including PNN [28], DRPNN [30], PanNet [40], ResTFNet [43], and TPNwFB [44].…”

Section: Simulated Experiments and Real Experimentsmentioning

confidence: 99%

See 2 more Smart Citations

A Dense Encoder–Decoder Network with Feedback Connections for Pan-Sharpening

Li¹,

Xiang²,

Liang³

2021

Remote Sensing

View full text Add to dashboard Cite

To meet the need for multispectral images having high spatial resolution in practical applications, we propose a dense encoder–decoder network with feedback connections for pan-sharpening. Our network consists of four parts. The first part consists of two identical subnetworks, one each to extract features from PAN and MS images, respectively. The second part is an efficient feature-extraction block. We hope that the network can focus on features at different scales, so we propose innovative multiscale feature-extraction blocks that fully extract effective features from networks of various depths and widths by using three multiscale feature-extraction blocks and two long-jump connections. The third part is the feature fusion and recovery network. We are inspired by the work on U-Net network improvements to propose a brand new encoder network structure with dense connections that improves network performance through effective connections to encoders and decoders at different scales. The fourth part is a continuous feedback connection operation with overfeedback to refine shallow features, which enables the network to obtain better reconstruction capabilities earlier. To demonstrate the effectiveness of our method, we performed several experiments. Experiments on various satellite datasets show that the proposed method outperforms existing methods. Our results show significant improvements over those from other models in terms of the multiple-target index values used to measure the spectral quality and spatial details of the generated images.

show abstract

Section: Cnn-based Pan-sharpeningmentioning

confidence: 99%

Section: Feedback Connection Structurementioning

confidence: 99%

See 1 more Smart Citation

A Dense Encoder–Decoder Network with Feedback Connections for Pan-Sharpening

Li¹,

Xiang²,

Liang³

2021

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…The spectral discriminators and the spatial discriminators are established to approximate the generated HRMS image consistent with the source images distribution. Besides, there are some other fusion models, such as the dense connection in DenseNet [32,33] the progressive up-sampling method [28], the feedback mechanism [34], etc. In conclusion, CNNs bring a new exploring direction and a superior fusion performance for pan-sharpening.…”

Section: Pan-sharpening Based On Cnnsmentioning

confidence: 99%

MESSFN : a Multi-level and Enhanced Spectral-Spatial Fusion Network for Pan-sharpening

Yuan,

Sun,

Zhang

2021

Preprint

View full text Add to dashboard Cite

Through fusing low resolution Multi-Spectral (MS) images and high resolution PANchromatic (PAN) images, pan-sharpening provides a satisfactory solution for the demands of high resolution multi-spectral images. However, dominant pan-sharpening frameworks simply concatenate the MS stream and the PAN stream once at a specific level. This way of fusion neglects the multi-level spectral-spatial correlation between the two streams, which is vital to improving the fusion performance. In consideration of this, we propose a Multi-level and Enhanced Spectral-Spatial Fusion Network (MESSFN) with the following innovations: First, to fully exploit and strengthen the above correlation, a Hierarchical Multi-level Fusion Architecture (HMFA) is carefully designed. A novel Spectral-Spatial (SS) stream is established to hierarchically derive and fuse the multi-level prior spectral and spatial expertise from the MS stream and the PAN stream. This helps the SS stream master a joint spectral-spatial representation in the hierarchical network for better modeling the fusion relationship. Second, to provide superior expertise, consequently, based on the intrinsic characteristics of the MS image and the PAN image, two feature extraction blocks are specially developed. In the MS stream, a Residual Spectral Attention Block (RSAB) is proposed to mine the potential spectral correlations between different spectra

show abstract

“…However, multi-spectral images can only be provided in low resolutions by optical remote sensing satellites. In the contribution by Fu et al [5] "Two-Path Network with Feedback Connections for Pan-Sharpening in Remote Sensing," authors present a new pan-sharpening technique which generates high-resolution multi-spectral images based on a panchromatic image and the low-resolution counterpart. The conventional deep learning based pan-sharpening methods process the panchromatic and the low-resolution image in a feedforward manner, where shallow layers fail to access useful information from deep layers.…”

Section: Applicationsmentioning

confidence: 99%

Editorial for the Special Issue “Advanced Machine Learning for Time Series Remote Sensing Data Analysis”

2020

Self Cite

View full text Add to dashboard Cite

This Special Issue intended to probe the impact of the adoption of advanced machine learning methods in remote sensing applications including those considering recent big data analysis, compression, multichannel, sensor and prediction techniques. In principal, this edition of the Special Issue is focused on time series data processing for remote sensing applications with special emphasis on advanced machine learning platforms. This issue is intended to provide a highly recognized international forum to present recent advances in time series remote sensing. After review, a total of eight papers have been accepted for publication in this issue.

show abstract

Two-Path Network with Feedback Connections for Pan-Sharpening in Remote Sensing

Cited by 16 publications

References 39 publications

A Dense Encoder–Decoder Network with Feedback Connections for Pan-Sharpening

A Dense Encoder–Decoder Network with Feedback Connections for Pan-Sharpening

MESSFN : a Multi-level and Enhanced Spectral-Spatial Fusion Network for Pan-sharpening

Editorial for the Special Issue “Advanced Machine Learning for Time Series Remote Sensing Data Analysis”

Contact Info

Product

Resources

About