Infrared and Visible Image Fusion Based on Gradient Transfer Optimization Model

Abstract

To tackle the problem of partial loss of image details in infrared and visible image fusion, a gradient transfer optimization model is proposed for the fusion of infrared and visible images. Firstly, an adaptive image decomposition method is proposed based on coupled partial differential equation, the infrared image and the visible image are decomposed into base layer and detail layer to extract the highbrightness target and the details of the two images. Based on this superior information of infrared image and visible image, the optimization model is designed to obtain the fusion image obvious target and rich details. For the proposed optimization model, Alternating Direction Method of Multipliers (ADMM) is used to decompose the original model into sub-problems that are easy to solve and iteratively optimize to obtain the optimal solution. The introduction of control parameters makes the model more flexible in different situations, and retains the thermal radiation information of the infrared image and the detailed information of the visible image to the greatest extent. The fused image visual effects and performance indicators are improved. We completed the experiment using a public data set and analyzed the experimental results. The experimental results show that the proposed method can better preserve the clear target and texture information of infrared and visible images, and the fusion results are more accurate and comprehensive. The experiment results also indicate that our method performs well and achieves comparable metric values with the state-of-the-art methods. INDEX TERMS Image fusion, optimization model, ADMM, partial differential equation.

ADF‐Net: Attention‐guided deep feature decomposition network for infrared and visible image fusion

“…To validate the effectiveness of the algorithm, we conducted comparative experiments with nine mainstream image fusion algorithms, namely, GTF [5], multi-scale transform and sparse representation (MST-SR) [6], FusionGAN [27], image fusion framework based convolutional neural network (IFCNN) [22], residual fusion network (RFN-Nest) [26], squeeze decomposition network (SDNet) [23], generative adversarial network multi classification constraints (GANMcC) [39], U2Fusion [40], and SeAFusion [34].…”

“…Yu et al. proposed a gradient transfer optimization model for the fusion of infrared and visible light images, which better addresses the issue of partial loss of image details in infrared and visible light image fusion [6]. Feng et al.…”

“…For example, Ma et al proposed a Gradient Transfer Fusion (GTF) algorithm, where the fusion problem is modelled as an L1-TV minimization problem [5]. Yu et al proposed a gradient transfer optimization model for the fusion of infrared and visible light images, which better addresses the issue of partial loss of image details in infrared and visible light image fusion [6]. Feng et al proposed an infrared and visible image fusion method based on two-scale decomposition and saliency extraction, using visualsaliencymap (VSM), Kirsch operator, and pre-trained Visual Geometry Group (VGG)19 network, but it is timeconsuming [7].…”

DFA-Net: Multi-Scale Dense Feature-Aware Network via Integrated Attention for Unmanned Aerial Vehicle Infrared and Visible Image Fusion

“…In order to obtain more accurate experimental results, we use 1083 images as the training set and 361 images as the test set. In this study, we compare our method with nine state-of-the-art approaches; these include two traditional methods, namely GTF [46] and MST-SR [47], an Ae-based method, RFN-Nest [48], two GAN-based methods, including FusionGAN and GANMcC [49], and four CNN-based approaches (namely IFCNN [50], U2Fusion [51], SDNet [39], and SeAFusion [41]). Finally, we tested our algorithm using data captured by a DJI Mavic 2 drone with a 12-megapixel camera sensor.…”

Reflectance estimation for infrared and visible image fusion

“…The dataset tested in this work contains 30 pairs of IR and VIS images from TNO dataset 1 containing aligned pairs of IR and VIS image. To explore the fusion performance of the proposed way, we compare it with 3 fusion methods, namely, two-scale image fusion based on visual saliency (TSIFVS) [35], GTF [25] and image fusion based on gradient transfer optimization model (GTOM) [36]. All of these methods are programmed in Matlab and parameters set goes along with original papers.…”