Dynamic Cross Feature Fusion for Remote Sensing Pansharpening

Wu, Xiao; Huang, Ting‐Zhu; Deng, Liang-Jian; Zhang, Tian-Jing

doi:10.1109/iccv48922.2021.01442

Cited by 43 publications

(9 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jin et al [28] designed a Laplacian pyramid pan-sharpening network (LPPN) under the Laplacian pyramid framework, which utilized the recursive structure to progressively fuse spatial information at different scales. Wu et al [29] proposed a dynamic cross feature fusion network (DCFNet). DCFNet contains a high-resolution branch served as the mainbranch and two parallel low-resolution branches to progressively supplement information to the mainbranch.…”

Section: A Related Cnn-based Pansharpening Methodsmentioning

confidence: 99%

Learning Correspondency in Frequency Domain by a Latent-Space Similarity Loss for Multispectral Pansharpening

Xing¹,

Zhang²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

The process of fuse a high spatial resolution (HR) panchromatic (PAN) image and a low spatial resolution (LR) multispectral (MS) image to obtain an HRMS image is known as pansharpening. With the development of convolutional neural networks, the performance of pansharpening methods has been improved, however, the blurry effects and the spectral distortion still exist in their fusion results due to the insufficiency in details learning and the mismatch between the high-frequency (HF) and low-frequency (LF) components. Therefore, the improvements of spatial details at the premise of reducing spectral distortion is still a challenge. In this paper, we propose a frequency-aware network (FAN) together with a novel latent-space similarity loss to address above mentioned problems. FAN is composed of three modules, where the frequency feature extraction module aims to extract features in the frequency domain with the help of discrete wavelet transform (DWT) layers, and the inverse DWT (IDWT) layers are then utilized in the frequency feature fusion module to reconstruct the features. Finally, the fusion results are obtained through the reconstruction module. In order to learn the correspondency, we also propose a latent-space similarity loss to constrain the LF features derived from PAN and MS branches, so that HF features of PAN can reasonably be used to supplement that of MS. Experimental results on three datasets at both reduced-and full-resolution demonstrate the superiority of the proposed method compared with several state-of-the-art pansharpening models, especially for the fusion at full resolution.

show abstract

Section: A Related Cnn-based Pansharpening Methodsmentioning

confidence: 99%

Learning Correspondency in Frequency Domain by a Latent-Space Similarity Loss for Multispectral Pansharpening

Xing¹,

Zhang²,

He³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…HRNet [11] maintains high-resolution representations in the forwarding propagation process by generating feature maps with different resolutions in parallel and repeatedly conducting multi-scale fusions in the exchange unit, which is friendly to dense prediction tasks. HRNet has been widely applied for human-pose estimation [11], [18], semantic segmentation [19], facial-landmark detection [16], surface-defect detection [20], video tracking [21], image inpainting [22], remote-sensing pansharpening [23], and gaze estimation [24].…”

Section: A High-resolution Networkmentioning

confidence: 99%

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Tang

Liu

Tan

et al. 2023

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

High-Resolution Transformer (HRFormer) can maintain high-resolution representation and share global receptive fields. It is friendly towards salient object detection (SOD) in which the input and output have the same resolution. However, two critical problems need to be solved for two-modality SOD. One problem is two-modality fusion. The other problem is the HRFormer output's fusion. To address the first problem, a supplementary modality is injected into the primary modality by using global optimization and an attention mechanism to select and purify the modality at the input level. To solve the second problem, a dual-direction short connection fusion module is used to optimize the output features of HRFormer, thereby enhancing the detailed representation of objects at the output level. The proposed model, named HRTransNet, first introduces an auxiliary stream for feature extraction of supplementary modality. Then, features are injected into the primary modality at the beginning of each multi-resolution branch. Next, HRFormer is applied to achieve forwarding propagation. Finally, all the output features with different resolutions are aggregated by intrafeature and inter-feature interactive transformers. Application of the proposed model results in impressive improvement for driving two-modality SOD tasks, e.g., RGB-D, RGB-T, and light field SOD.https://github.com/liuzywen/HRTransNet

show abstract

“…Finally, through a simple convolution layer output, this is a complete introduction to the multi-scale residual space spectrum attention module, whose structure is shown in Figure 2. For the information injection module from high-resolution branch to low-resolution branch, dynamic weight addition [38] is designed as shown in Formula (1), whose structure is similar to the Softmax function, that is the proportion of each input to the total input is calculated, and then weighted addition is performed.…”

Section: A Multistage Super-resolution Modulementioning

confidence: 99%

Multistage Progressive Interactive Fusion Network for Sentinel-2: High Resolution for All Bands

Liu,

Meng,

Liu

et al. 2023

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Sentinel-2 satellite remote sensing images have been widely used in various fields, such as change detection and resource monitoring. However, Sentinel-2 provides multispectral bands with inconsistent spatial resolutions (i.e., 60 m for three bands, 20 m for six bands, and 10 m for four bands), which has greatly limited the application values, especially for cooperative analysis or application on different bands at a unified resolution. In this paper, we proposed a Multistage Progressive Interactive Fusion Network (MPIFNet) to generate all 10 m high-resolution bands. Specifically, a refined multi-stage spatial resolution enhancement model in a multistage way is developed to progressively improve the low-resolution bands, to preserve the spectral information of the enhanced bands. Moreover, an information interaction module is proposed for the three branches of high (10 m), medium (20 m) and low-resolution (60 m) bands to achieve effective information interaction. The experimental results show that our method is superior to other existing state-of-the-art methods, and it can be applied to the reconstruction of the high-resolution vegetation index.

show abstract

Dynamic Cross Feature Fusion for Remote Sensing Pansharpening

Cited by 43 publications

References 25 publications

Learning Correspondency in Frequency Domain by a Latent-Space Similarity Loss for Multispectral Pansharpening

Learning Correspondency in Frequency Domain by a Latent-Space Similarity Loss for Multispectral Pansharpening

HRTransNet: HRFormer-Driven Two-Modality Salient Object Detection

Multistage Progressive Interactive Fusion Network for Sentinel-2: High Resolution for All Bands

Contact Info

Product

Resources

About