Improving the Performance of Image Fusion Based on Visual Saliency Weight Map Combined With CNN

Abstract

Convolutional neural networks (CNN) with their deep feature extraction capability have recently been applied in numerous image fusion tasks. However, the image fusion of infrared and visible images leads to loss of fine details and degradation of contrast in the fused image. This deterioration in the image is associated with the conventional ''averaging'' rule for base layer fusion and relatively large feature extraction by CNN. To overcome these problems, an effective fusion framework based on visual saliency weight map (VSWM) combined with CNN is proposed. The proposed framework first employs VSWM method to improve the contrast of an image under consideration. Next, the fine details in the image are preserved by applying multi-resolution singular value decomposition (MSVD) before further processing by CNN. The promising experimental results show that the proposed method outperforms state-of-the-art methods by scoring the highest over different evaluation metrics such as Q 0 , multiscale structural similarity (MS_SSIM), and the sum of correlations of differences (SCD). INDEX TERMS Convolutional neural network, image fusion, visual saliency weight map, multi-resolution singular value decomposition.

FSADFuse: A Novel Fusion Approach to Infrared and Visible Images

“…In order to objectively and quantitatively evaluate the performances of the different models and fusion methods, shannon entropy (EN), MEAN, standard deviation (SD) [40], joint entropy (JE) [41], (NSS) [42]and cross entropy (SCD) [43] were selected as objective evaluation indicators. The aggregation feature of the image's gray distribution is reflected by EN.…”