IPLF: A Novel Image Pair Learning Fusion Network for Infrared and Visible Image

Zhu, Daming; Zhan, Weida; Jiang, Yichun; Xu, Xiaoyu; Guo, Ruifeng

doi:10.1109/jsen.2022.3161733

Cited by 27 publications

(9 citation statements)

References 49 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…FusionGAN achieves adversarial learning through a generator and discriminator based on local receptive fields. Similarly, some other CNN-based methods 19 , 35 , 36 also rely on feature extraction from local receptive fields, neglecting the long-range dependency of features. The absence of features extracted from larger receptive fields results in the loss of contextual information, leading to a lack of overall consistency in the fused image.…”

Section: Related Workmentioning

confidence: 99%

MCFusion: infrared and visible image fusion based multiscale receptive field and cross-modal enhanced attention mechanism

Jiang,

Wang,

Kong

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Infrared and visible image fusion aims to generate a more comprehensive image for improving measurement and analysis accuracy. Existing Swin Transformer based methods effectively extract features from medium-large scale receptive fields. However, these methods ignore the complementary role of features extracted from the small receptive field, resulting in the loss of essential fine information. To this end, we propose MCFusion, a dual-branch framework based on a multiscale receptive field and cross-modal enhanced attention mechanism. MCFusion is composed of an attention-guided coarse branch (AGCB) and a fine branch (FineB). First, AGCB and FineB are designed to extract distinct yet complementary features from medium-large and small receptive fields, respectively. Second, a cross-modal enhanced attention mechanism is designed to enhance shared features across different modalities. Third, a loss function fully considering contrast, texture, and illumination intensity is proposed to generate fused images. The test results demonstrate that MCFusion is superior to other state-of-the-art methods and can notably enhance the detection and recognition of targets.

show abstract

Section: Related Workmentioning

confidence: 99%

MCFusion: infrared and visible image fusion based multiscale receptive field and cross-modal enhanced attention mechanism

Jiang,

Wang,

Kong

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Comparisons and Evaluation: Seven state-of-the-art methods are selected to compare with our proposed methods, namely, SDNet [54], SwinFusion [21], GANMcC [25], U2Fusion [55], UNFusion [19], IPLF [20] and CDDFuse [43]. The objective metrics selected are entropy (EN) [56], standard deviation (SD) [57], edge intensity (EI) [58], visual information fidelity (VIF) [59], Chen-Blum metric (Q CB ) [60] and MS-SSIM [61].…”

Section: Datasets and Pre-processingmentioning

confidence: 99%

Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition

Wang,

Shu,

et al. 2024

IEEE Access

View full text Add to dashboard Cite

Infrared and visible image fusion (IVIF) aims to generate a fused image with both salient target and rich textures from two different complementary modality images. To better integrate valuable edge information into the fused image, we first propose a novel two-stream network based on Auto-Encoder (AE) framework, which extracts deep hierarchical detail information at coarse scale from base stream by multi-level wavelet decomposition progressively and incorporates them into detail stream for information compensation. The aggregation of edge information ranging from coarse to fine facilitates a more comprehensive representation of contours and textures. Then, we propose a new feature fusion strategy, termed as Structural Feature Map Decomposition (SFMD). The first step is to decompose local patches of feature map with each modality into three independent components by Structural Patch Decomposition (SPD). In the second step, appropriate fusion rules are carefully designed for each component and the fused patch can be derived by inverse SPD. Our extensive experiments on several benchmark datasets show that our method outperforms seven compared state-of-the-art methods, especially in human visual perception.

show abstract

“…Thermal infrared imaging is a passive imaging technology that detects the thermal radiation passively emitted by objects to form an image [ 1 ]. It has the advantages of strong anti-interference ability and the capability to distinguish between targets and backgrounds.…”

Section: Introductionmentioning

confidence: 99%

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Jiang

Liu

Zhan

et al. 2023

Entropy

Self Cite

View full text Add to dashboard Cite

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network’s ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

show abstract

IPLF: A Novel Image Pair Learning Fusion Network for Infrared and Visible Image

Cited by 27 publications

References 49 publications

MCFusion: infrared and visible image fusion based multiscale receptive field and cross-modal enhanced attention mechanism

MCFusion: infrared and visible image fusion based multiscale receptive field and cross-modal enhanced attention mechanism

Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion

Contact Info

Product

Resources

About