A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

Abstract

In multi-modality image fusion, source image decomposition, such as multi-scale transform (MST), is a necessary step and also widely used. However, when MST is directly used to decompose source images into high-and low-frequency components, the corresponding decomposed components are not precise enough for the following infrared-visible fusion operations. This paper proposes a non-subsampled contourlet transform (NSCT) based decomposition method for image fusion, by which source images are decomposed to obtain corresponding high-and low-frequency sub-bands. Unlike MST, the obtained high-frequency sub-bands have different decomposition layers, and each layer contains different information. In order to obtain a more informative fused high-frequency component, maximum absolute value and pulse coupled neural network (PCNN) fusion rules are applied to different sub-bands of high-frequency components. Activity measures, such as phase congruency (PC), local measure of sharpness change (LSCM), and local signal strength (LSS), are designed to enhance the detailed features of fused low-frequency components. The fused high-and low-frequency components are integrated to form a fused image. The experiment results show that the fused images obtained by the proposed method achieve good performance in clarity, contrast, and image information entropy.Entropy 2019, 21, 1135 2 of 16 for infrared-visible image fusion, have been widely applied to the target recognition in different environments, such as smart city, battlefield, remote sensing, and so on [2,3].In recent years, transform domain based methods have become the mainstream in infrared-visible image fusion, which include pyramid, wavelet transform, multi-scale geometric transform, sparse representation [4,5], and so on. Pyramid, wavelet transform, and multi-scale geometric transform can be categorized as MST-based methods. MST-based methods have three main steps. First, MST is employed to decompose each source image into high-frequency sub-bands at different scales and directions as well as one low-frequency sub-band. Then, the obtained high-and low-frequency sub-bands are fused separately following different fusion rules. Finally, the fused image is obtained by performing the inverse MST on both fused high-and low-frequency sub-bands. Double-tree complex wavelet transform as a kind of wavelet transform can only capture a limited amount of edge information, but cannot correctly and effectively represent the discontinuity of lines and curves [6]. As a true two-dimensional (2D) multi-scale geometric analysis method, contourlet transform (CT) possesses localization, multi-resolution, multi-scale, multi-direction, and anisotropy.As a shift-invariant version of CT, NSCT performs well in transform domain, and has been widely used in image fusion. NSCT has multi-scale and multi-direction features, which can solve the limitations of traditional wavelet methods in the representation of image curves and edges [7]. Compared with traditional MST-based image fusion methods, NSCT has shif...

Entropy-Based Algorithms for Signal Processing

“…However, when MST is directly applied to decompose source images into high- and low-frequency components, the corresponding decomposed components are not precise enough for the following infrared-visible fusion operations. The contribution by Huang et al [ 9 ], “A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy,” proposes a non-subsampled contourlet transform based decomposition method for image fusion, by which source images are decomposed to obtain corresponding high- and low-frequency sub-bands. Unlike MST, the obtained high-frequency sub-bands have different decomposition layers, and each layer contains different information.…”

A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation

“…Relatively, reflected light captured by visible light sensors could be represented in visible images; hence, visible images can provide texture details with high spatial resolution and definition, in a manner consistent with the human visual system [ 4 , 5 ]. Therefore, fused images can simultaneously preserve the texture details of visible images and the contrast of infrared images, benefiting subsequent tasks [ 6 , 7 , 8 ].…”

Improving the Performance of Infrared and Visible Image Fusion Based on Latent Low-Rank Representation Nested With Rolling Guided Image Filtering

“…In recent years, to improve the fusion quality and obtain clearer scene information and better target features, researchers have devoted themselves to the research of new infrared and visible image fusion methods, and a variety of infrared and visible image fusion methods have been proposed [8][9][10][11][12][13][14][15][16][17][18][19]. To overcome the shortcomings of methods based on MST and SR, Li et al [8] proposed an image fusion framework integrating nonsubsampled contour transformation (NSCT) into sparse representation.…”