Unified framework based on multiscale transform and feature learning for infrared and visible image fusion

Fan, Zunlin; Guan, Naiyang; Wang, Zhiyuan; Su, Lili; Jian-gang, WU; Sun, Qianchong

doi:10.1117/1.oe.60.12.123102

Cited by 5 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This mechanism helps humans suppress useless information and quickly obtain information on key areas. The attention mechanism in computer vision is comparable to humans, focusing on key regions in images [27]. The visual attention mechanism based on deep learning is implemented through a mask mechanism, using weights to mark important features of the image, and forming attention through neural network learning.…”

Section: Artistic Image Style Conversion Based On Introduced Attentio...mentioning

confidence: 99%

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

Li,

Zhu

2024

IJCAI

View full text Add to dashboard Cite

Section: Artistic Image Style Conversion Based On Introduced Attentio...mentioning

confidence: 99%

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

Li,

Zhu

2024

IJCAI

View full text Add to dashboard Cite

“…Early deep learning methods [16][17][18][19][20] showcased impressive feature extraction capabilities of neural networks but struggled with effectively extracting multi-scale features. To overcome the need for human intervention in designing fusion strategies, the end-to-end methods [21][22][23] emerged, treating the task as a whole and directly mapping input to output. In image fusion, Ma et al 24 first introduced generative adversarial network (GAN), which adjusts the visible information in a fusion image using a discriminator.…”

Section: Introductionmentioning

confidence: 99%

“…E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 2 1 ; 1 1 7 ; 9 5 ∇F y ði; jÞ ¼ Fði; jÞ − Fði; j þ 1Þ:(21) …”

unclassified

SMFD: an end-to-end infrared and visible image fusion model based on shared-individual multi-scale feature decomposition

Xu,

Kong,

Jiang

et al. 2024

J. Appl. Rem. Sens.

View full text Add to dashboard Cite

By leveraging the characteristics of different optical sensors, infrared and visible image fusion generates a fused image that combines prominent thermal radiation targets with clear texture details. Existing methods often focus on a single modality or treat two modalities equally, which overlook the distinctive characteristics of each modality and fail to fully utilize their complementary information. To address this problem, we propose an end-to-end infrared and visible image fusion model based on shared-individual multi-scale feature decomposition. First, to extract multi-scale features from source images, a symmetric multi-scale decomposition encoder consisting of nest connections and a multi-scale receptive field network is designed to capture small, medium, and large-scale features. Second, to sufficiently utilize complementary information, common edge feature maps are introduced to the feature decomposition loss function to decompose extracted features into shared and individual features. Third, to aggregate shared and individual features, a sharedindividual self-augmented decoder is proposed to take the individual fusion feature maps as the main input and the shared fusion feature maps as the residual input to assist the decoding process and the reconstruct the fused image. Finally, through comparing subjective evaluations and objective metrics, our method demonstrates its superiority compared with the state-of-the-art approaches.

show abstract

“…Visible (low-light-level) images contain rich scene detail and texture information, infrared images can reflect the temperature information of an object and are less affected by weather and lighting conditions 1 . Visible (low-light-level) and infrared color night vision fusion technology combines the advantages of the two bands to produce false color fusion images with obvious hot targets and rich details 2 . The fusion images are suitable for human observation and have wide applications in military target detection, remote sensing, situation awareness, and many other fields 3 , 4 .…”

Section: Introductionmentioning

confidence: 99%

“…1 Visible (low-light-level) and infrared color night vision fusion technology combines the advantages of the two bands to produce false color fusion images with obvious hot targets and rich details. 2 The fusion images are suitable for human observation and have wide applications in military target detection, remote sensing, situation awareness, and many other fields. 3,4 However, using different color fusion methods to fuse the same set of source images may obtain quite different fusion effects, which directly affects the observer's understanding of the image.…”

Section: Introductionmentioning

confidence: 99%

Research on objective assessment of dual-band color night vision fusion image quality based on scene understanding

Gao,

Yi,

Tian

et al. 2023

Opt. Eng.

View full text Add to dashboard Cite

The color quality and sharpness of the dual-band color fusion images have the greatest impact on the audience's understanding of the scene content. Traditional image quality assessment (IQA) methods focus on the luminance contrast of image and attach limited attention to pixel-wise relationship. As a result, these methods perform poorly on color-related distortions. A objective assessment model for the comprehensive quality of fusion images is proposed. According to the correlation between the gray level co-occurrence matrix (GLCM) and the image color, the model extracts the homogeneity, angular second moment, and contrast of the GLCM as the image color quality features. Gradient-weighted class activation mapping is used to calculate the heat map, reflecting the visual saliency of each region. We use the visual saliency weight of each region to weight the rotation and uniform invariant local binary pattern histograms of the corresponding regions to obtain sharpness features. The experimental results show that the performance of the proposed method is better than the existing eight no-reference IQA models.

show abstract

Unified framework based on multiscale transform and feature learning for infrared and visible image fusion

Cited by 5 publications

References 0 publications

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

Art Image Style Conversion Based on Multi-Scale Feature Fusion Network

SMFD: an end-to-end infrared and visible image fusion model based on shared-individual multi-scale feature decomposition

Research on objective assessment of dual-band color night vision fusion image quality based on scene understanding

Contact Info

Product

Resources

About