RFN-Nest: An end-to-end residual fusion network for infrared and visible images

Li, Hui; Wu, Xiao-Jun; Kittler, Josef

doi:10.1016/j.inffus.2021.02.023

Cited by 555 publications

(218 citation statements)

References 69 publications

Supporting

Mentioning

218

Contrasting

Order By: Relevance

“…In order to test the practicability and effectiveness of the proposed method, we set up two groups of experiments. The first group compares the proposed method with RE-NSST and IFS-NSST methods, and the second group compares the proposed method with the other nine advanced fusion methods which are FPDE [ 35 ] (fourth-order partial differential equations), VSM [ 36 ] (visual saliency map), Bala [ 37 ] (Bala fuzzy sets), Gauss [ 34 ] (Gauss fuzzy sets), DRTV [ 38 ] (Different Resolutions via Total Variation Model), LATLRR [ 39 ] (latent Low-Rank Representation), SR [ 40 ] (sparse regularization), MDLatLRR [ 41 ] (decomposition method based on latent low-rank representation) and RFN-Nest [ 42 ] (residual fusion network).…”

Section: Resultsmentioning

confidence: 99%

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion

Xing

Luo

Zhou

et al. 2021

Sensors

View full text Add to dashboard Cite

To get more obvious target information and more texture features, a new fusion method for the infrared (IR) and visible (VIS) images combining regional energy (RE) and intuitionistic fuzzy sets (IFS) is proposed, and this method can be described by several steps as follows. Firstly, the IR and VIS images are decomposed into low- and high-frequency sub-bands by non-subsampled shearlet transform (NSST). Secondly, RE-based fusion rule is used to obtain the low-frequency pre-fusion image, which allows the important target information preserved in the resulting image. Based on the pre-fusion image, the IFS-based fusion rule is introduced to achieve the final low-frequency image, which enables more important texture information transferred to the resulting image. Thirdly, the ‘max-absolute’ fusion rule is adopted to fuse high-frequency sub-bands. Finally, the fused image is reconstructed by inverse NSST. The TNO and RoadScene datasets are used to evaluate the proposed method. The simulation results demonstrate that the fused images of the proposed method have more obvious targets, higher contrast, more plentiful detailed information, and local features. Qualitative and quantitative analysis results show that the presented method is superior to the other nine advanced fusion methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion

Xing

Luo

Zhou

et al. 2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Our method is based on the PyTorch framework and runs on a NVIDIA GTX1070 GPU. We evaluated the effectiveness and efficiency of our scheme by comparing with a number of recent CNN-based methods (i.e., PMGI [50], FGAN [37], DDcGAN [36], FusionDN [46], U2Fusion [45], RFN [19], GANMcC [38], Nest [16], DID [51] and Dense [15]).…”

Section: Training Detailsmentioning

confidence: 99%

Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion

Liu

et al. 2021

Proceedings of the 29th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Multi-modality image fusion refers to generating a complementary image that integrates typical characteristics from source images. In recent years, we have witnessed the remarkable progress of deep learning models for multi-modality fusion. Existing CNN-based approaches strain every nerve to design various architectures for realizing these tasks in an end-to-end manner. However, these handcrafted designs are unable to cope with the high demanding fusion tasks, resulting in blurred targets and lost textural details. To alleviate these issues, in this paper, we propose a novel approach, aiming at searching effective architectures according to various modality principles and fusion mechanisms. Specifically, we construct a hierarchically aggregated fusion architecture to extract and refine fused features from feature-level and object-level fusion perspectives, which is responsible for obtaining complementary target/detail representations. Then by investigating diverse effective practices, we composite a more flexible fusion-specific search space. Motivated by the collaborative principle, we employ a new search strategy with different principled losses and hardware constraints for sufficient discovery of components. As a result, we can obtain a task-specific architecture with fast inference time. Extensive quantitative and qualitative results demonstrate the superiority and versatility of our method against state-of-the-art methods. CCS CONCEPTS• Computing methodologies → Computer vision.

show abstract

“…Thus, in addition to fusion quality metrics, we added a perceptual quality metric to show the performance of our method more accurately. Furthermore, we conducted an experiment on TNO [13] dataset using DLF [2], DeepFuse [3], Dual Branch [6] and Rfn-Nest [5] state-of-the-art deep learning methods and Hybrid MSD [16] method. In Table 1, average scores of the six methods, including the proposed method, are given and it is seen that our proposed method obtain the best results.…”

Section: Quantitative Performance Comparisonsmentioning

confidence: 99%

“…This method is used to solve semantic segmentation problems in multi-spectral image pairs. The latest approaches Rfn-Nest [5] and Dual-Branch [6] also use the power of the encoder-decoder network. They directly reconstruct the fused image from source image pairs.…”

Section: Introductionmentioning

confidence: 99%

Visible And Infrared Image Fusion Using Encoder-Decoder Network

Ataman¹,

Akar²

2021

2021 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

The aim of multispectral image fusion is to combine object or scene features of images with different spectral characteristics to increase the perceptual quality. In this paper, we present a novel learning-based solution to image fusion problem focusing on infrared and visible spectrum images. The proposed solution utilizes only convolution and pooling layers together with a loss function using no-reference quality metrics. The analysis is performed qualitatively and quantitatively on various datasets. The results show better performance than stateof-the-art methods. Also, the size of our network enables real-time performance on embedded devices. Project codes can be found at https://github.com/ferhatcan/ pyFusionSR.

show abstract

RFN-Nest: An end-to-end residual fusion network for infrared and visible images

Cited by 555 publications

References 69 publications

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion

Combining Regional Energy and Intuitionistic Fuzzy Sets for Infrared and Visible Image Fusion

Searching a Hierarchically Aggregated Fusion Architecture for Fast Multi-Modality Image Fusion

Visible And Infrared Image Fusion Using Encoder-Decoder Network

Contact Info

Product

Resources

About