Infrared and visible image fusion method based on saliency detection in sparse domain

Liu, C.H.; Qi, Yue; Ding, Wenrui

doi:10.1016/j.infrared.2017.04.018

Cited by 231 publications

(113 citation statements)

References 24 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…These layers are relu 1 1, relu 2 1, relu 3 1 and relu 4 1, respectively For comparison, we selected several recent and classical fusion methods to perform the same experiment, including: cross bilateral filter fusion method(CBF) [21], the joint-sparse representation model(JSR) [10], the JSR model with saliency detection fusion method(JSRSD) [22], weighted least square optimization-based method(WLS) [20] and the convolutional sparse representation model(ConvSR) [13].…”

Section: A Experimental Settingsmentioning

confidence: 99%

Infrared and Visible Image Fusion using a Deep Learning Framework

Li¹,

Wu²,

Kittler³

2018

2018 24th International Conference on Pattern Recognition (ICPR)

424

276

View full text Add to dashboard Cite

In recent years, deep learning has become a very active research tool which is used in many image processing fields. In this paper, we propose an effective image fusion method using a deep learning framework to generate a single image which contains all the features from infrared and visible images. First, the source images are decomposed into base parts and detail content. Then the base parts are fused by weighted-averaging. For the detail content, we use a deep learning network to extract multi-layer features. Using these features, we use l1-norm and weighted-average strategy to generate several candidates of the fused detail content. Once we get these candidates, the max selection strategy is used to get the final fused detail content. Finally, the fused image will be reconstructed by combining the fused base part and the detail content. The experimental results demonstrate that our proposed method achieves stateof-the-art performance in both objective assessment and visual quality. The Code of our fusion method is available at https: //github.com/hli1221/imagefusion deeplearning.

show abstract

Section: A Experimental Settingsmentioning

confidence: 99%

Infrared and Visible Image Fusion using a Deep Learning Framework

Li¹,

Wu²,

Kittler³

2018

2018 24th International Conference on Pattern Recognition (ICPR)

424

276

View full text Add to dashboard Cite

show abstract

“…In Equation (1), H l,k F (i, j) represents the fused high-frequency coefficients. H l,k F (i, j) l=5 represents the 5-level high-frequency fusion coefficients, which can be obtained by Equation (2). Equation (2) integrates the PCNN model, in which the entropy of the absolute value of high-frequency band is used as the network input.…”

Section: As Shown In Equationmentioning

confidence: 99%

“…The fused high-and low-frequency components are integrated to form a fused image. The experiment results show that the fused images obtained by the proposed method achieve good performance in clarity, contrast, and image information entropy.Entropy 2019, 21, 1135 2 of 16 for infrared-visible image fusion, have been widely applied to the target recognition in different environments, such as smart city, battlefield, remote sensing, and so on [2,3].In recent years, transform domain based methods have become the mainstream in infrared-visible image fusion, which include pyramid, wavelet transform, multi-scale geometric transform, sparse representation [4,5], and so on. Pyramid, wavelet transform, and multi-scale geometric transform can be categorized as MST-based methods.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

Huang

Wei

et al. 2019

Entropy

View full text Add to dashboard Cite

In multi-modality image fusion, source image decomposition, such as multi-scale transform (MST), is a necessary step and also widely used. However, when MST is directly used to decompose source images into high-and low-frequency components, the corresponding decomposed components are not precise enough for the following infrared-visible fusion operations. This paper proposes a non-subsampled contourlet transform (NSCT) based decomposition method for image fusion, by which source images are decomposed to obtain corresponding high-and low-frequency sub-bands. Unlike MST, the obtained high-frequency sub-bands have different decomposition layers, and each layer contains different information. In order to obtain a more informative fused high-frequency component, maximum absolute value and pulse coupled neural network (PCNN) fusion rules are applied to different sub-bands of high-frequency components. Activity measures, such as phase congruency (PC), local measure of sharpness change (LSCM), and local signal strength (LSS), are designed to enhance the detailed features of fused low-frequency components. The fused high-and low-frequency components are integrated to form a fused image. The experiment results show that the fused images obtained by the proposed method achieve good performance in clarity, contrast, and image information entropy.Entropy 2019, 21, 1135 2 of 16 for infrared-visible image fusion, have been widely applied to the target recognition in different environments, such as smart city, battlefield, remote sensing, and so on [2,3].In recent years, transform domain based methods have become the mainstream in infrared-visible image fusion, which include pyramid, wavelet transform, multi-scale geometric transform, sparse representation [4,5], and so on. Pyramid, wavelet transform, and multi-scale geometric transform can be categorized as MST-based methods. MST-based methods have three main steps. First, MST is employed to decompose each source image into high-frequency sub-bands at different scales and directions as well as one low-frequency sub-band. Then, the obtained high-and low-frequency sub-bands are fused separately following different fusion rules. Finally, the fused image is obtained by performing the inverse MST on both fused high-and low-frequency sub-bands. Double-tree complex wavelet transform as a kind of wavelet transform can only capture a limited amount of edge information, but cannot correctly and effectively represent the discontinuity of lines and curves [6]. As a true two-dimensional (2D) multi-scale geometric analysis method, contourlet transform (CT) possesses localization, multi-resolution, multi-scale, multi-direction, and anisotropy.As a shift-invariant version of CT, NSCT performs well in transform domain, and has been widely used in image fusion. NSCT has multi-scale and multi-direction features, which can solve the limitations of traditional wavelet methods in the representation of image curves and edges [7]. Compared with traditional MST-based image fusion methods, NSCT has shif...

show abstract

“…Especially in military field, infrared (IR) and visible (VI) image fusion is important to military science technology, such as automatic military target detection and localization. As a hot image fusion field, it has attracted the attention of many researchers [1][2][3][4][5][6][7]. The key problem of IR and VI image fusion is to integrate and extract the feature information of the source images to produce a new image which is more reliable and understandable, and the fused image not only has the detailed texture information of VI image but also can highlight the target area in an IR image.…”

Section: Introductionmentioning

confidence: 99%

Infrared and Visible Image Fusion Combining Interesting Region Detection and Nonsubsampled Contourlet Transform

Zhou

Zhang

et al. 2018

Journal of Sensors

View full text Add to dashboard Cite

The most fundamental purpose of infrared (IR) and visible (VI) image fusion is to integrate the useful information and produce a new image which has higher reliability and understandability for human or computer vision. In order to better preserve the interesting region and its corresponding detail information, a novel multiscale fusion scheme based on interesting region detection is proposed in this paper. Firstly, the MeanShift is used to detect the interesting region with the salient objects and the background region of IR and VI. Then the interesting regions are processed by the guided filter. Next, the nonsubsampled contourlet transform (NSCT) is used for background region decomposition of IR and VI to get a low-frequency and a series of high-frequency layers. An improved weighted average method based on per-pixel weighted average is used to fuse the low-frequency layer. The pulse-coupled neural network (PCNN) is used to fuse each high-frequency layer. Finally, the fused image is obtained by fusing the fused interesting region and the fused background region. Experimental results demonstrate that the proposed algorithm can integrate more background details as well as highlight the interesting region with the salient objects, which is superior to the conventional methods in objective quality evaluations and visual inspection.

show abstract

Infrared and visible image fusion method based on saliency detection in sparse domain

Cited by 231 publications

References 24 publications

Infrared and Visible Image Fusion using a Deep Learning Framework

Infrared and Visible Image Fusion using a Deep Learning Framework

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

Infrared and Visible Image Fusion Combining Interesting Region Detection and Nonsubsampled Contourlet Transform

Contact Info

Product

Resources

About