A Cross-Scale Iterative Attentional Adversarial Fusion Network for Infrared and Visible Images

Wang, Zhishe; Shao, Wenyu; Chen, Yanlin; Xu, Jiawei; Zhang, Lei

doi:10.1109/tcsvt.2023.3239627

Cited by 38 publications

(4 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Due to the powerful nonlinear fitting capabilities, neural networks have been widely applied in infrared and visible image fusion, achieving performance far superior to traditional methods. Currently, the methods of infrared and visible image fusion based on deep learning can generally be divided into four types: CNN-based methods [ 33 , 34 , 35 , 36 ], GAN-based methods [ 37 , 38 , 39 , 40 , 41 ], AE-based [ 1 , 42 , 43 , 44 , 45 ] methods, and transformer-based [ 32 , 46 , 47 , 48 , 49 ] methods. CNN-based methods tend to focus on the design of loss functions, forcing the model to generate images that contain as much information from the source images as possible.…”

Section: Related Workmentioning

confidence: 99%

FERFusion: A Fast and Efficient Recursive Neural Network for Infrared and Visible Image Fusion

Yang,

Xiang,

Chen

et al. 2024

Sensors

View full text Add to dashboard Cite

The rapid development of deep neural networks has attracted significant attention in the infrared and visible image fusion field. However, most existing fusion models have many parameters and consume high computational and spatial resources. This paper proposes a fast and efficient recursive fusion neural network model to solve this complex problem that few people have touched. Specifically, we designed an attention module combining a traditional fusion knowledge prior with channel attention to extract modal-specific features efficiently. We used a shared attention layer to perform the early fusion of modal-shared features. Adopting parallel dilated convolution layers further reduces the network’s parameter count. Our network is trained recursively, featuring minimal model parameters, and requires only a few training batches to achieve excellent fusion results. This significantly reduces the consumption of time, space, and computational resources during model training. We compared our method with nine SOTA methods on three public datasets, demonstrating our method’s efficient training feature and good fusion results.

show abstract

Section: Related Workmentioning

confidence: 99%

FERFusion: A Fast and Efficient Recursive Neural Network for Infrared and Visible Image Fusion

Yang,

Xiang,

Chen

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…where ∇ denotes the laplacian operator, max() refers to the element-wise maximum selection. 4) Total loss function: Our method jointly trains the visible image enhancement module and image fusion module, so the total loss is the weighted sum of all sub-losses mentioned before, which is expressed as: (18) where, λ t denotes the coefficients corresponding to different sub-loss functions. Empirically, we set λ illu = 10, λ col = 6, λ tv = 200, λ str = 1, λ grad = 1.…”

Section: Loss Functionsmentioning

confidence: 99%

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

Zhang,

Wang,

Yan

et al. 2024

IEEE Sensors J.

View full text Add to dashboard Cite

Infrared and visible image fusion can effectively integrate the advantages of two source images, preserving significant target information and rich texture details. However, most existing fusion methods are only designed for well-illuminated scenes and tend to lose details when encountering low-light scenes because of the poor brightness of visible images. Some methods incorporate a light adjustment module, but they typically focus only on enhancing intensity information and neglect the enhancement of color feature, resulting in unsatisfactory visual effects in the fused images. To address this issue, this paper proposes a novel method called EVfusion, which explores the potential color and detail features in visible images and improve the visual perception of fused images. Specifically, an unsupervised image enhancement module is designed that effectively restores texture, structure and color information in visible images by several non-reference loss functions. Then, an intensity image fusion module is devised to integrate the enhanced visible image and the infrared image. Moreover, to improve the infrared salient object feature in the fused images, we propose an infrared bilateral-guided salience map embedding into the fusion loss functions. Extensive experiments demonstrate that our method outperforms state-of-the-art infrared visible image fusion methods.

show abstract

“…To comprehensively evaluate the performance of our fusion model, we compare it with nine methods, which include traditional fusion methods (CBF [15], LP [16]), CNN-based methods (IFCNN [1], SeAFusion [31], SEDRFuse [54]), GANbased methods (FusionGAN [34], DDcGAN [35], CrossFuse [55]) and transformer-based methods (SwinFusion [2]). We first conduct the comparison experiments on the TNO dataset, and both infrared and visible images in TNO are gray images.…”

Section: A Experimental Configurationsmentioning

confidence: 99%

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Li,

Yang,

Bai

et al. 2023

IEEE Trans. Instrum. Meas.

View full text Add to dashboard Cite

Infrared and visible image fusion aims to extract complementary features to synthesize a single fused image. Many methods employ convolutional neural networks (CNNs) to extract local features due to its translation invariance and locality. However, CNNs fail to consider the image's non-local self-similarity (NLss), though it can expand the receptive field by pooling operations, it still inevitably leads to information loss. In addition, the transformer structure extracts long-range dependence by considering the correlativity among all image patches, leading to information redundancy of such transformer-based methods. However, graph representation is more flexible than grid (CNN) or sequence (transformer structure) representation to address irregular objects, and graph can also construct the relationships among the spatially repeatable details or texture with far-space distance. Therefore, to address the above issues, it is significant to convert images into the graph space and thus adopt graph convolutional networks (GCNs) to extract NLss. This is because the graph can provide a fine structure to aggregate features and propagate information across the nearest vertices without introducing redundant information. Concretely, we implement a cascaded NLss extraction pattern to extract NLss of intra-and inter-modal by exploring interactions of different image pixels in intra-and inter-image positional distance. We commence by preforming GCNs on each intra-modal to aggregate features and propagate information to extract independent intra-modal NLss. Then, GCNs are performed on the concatenate intramodal NLss features of infrared and visible images, which can explore the cross-domain NLss of inter-modal to reconstruct the fused image. We progressively expand the kennel sizes and dilations to increase the receptive field of GCNs, when we extract intra-and inter-modal NLss. Ablation studies and extensive experiments illustrates the effectiveness and superiority of the proposed method on three datasets.

show abstract

A Cross-Scale Iterative Attentional Adversarial Fusion Network for Infrared and Visible Images

Cited by 38 publications

References 42 publications

FERFusion: A Fast and Efficient Recursive Neural Network for Infrared and Visible Image Fusion

FERFusion: A Fast and Efficient Recursive Neural Network for Infrared and Visible Image Fusion

EV-Fusion: A Novel Infrared and Low-Light Color Visible Image Fusion Network Integrating Unsupervised Visible Image Enhancement

TFIV: Multigrained Token Fusion for Infrared and Visible Image via Transformer

Contact Info

Product

Resources

About