A novel multi-modality image fusion method based on image decomposition and sparse representation

Zhu, Zhiqin; Yin, Hongpeng; Chai, Yi; Li, Yanxia; Qi, Guanqiu

doi:10.1016/j.ins.2017.09.010

Cited by 329 publications

(167 citation statements)

References 44 publications

Supporting

Mentioning

167

Contrasting

Order By: Relevance

“…In this section, the proposed NSCT-based fusion framework is compared with seven popular fusion methods, such as the adaptive spare representation (ASR) based image fusion method proposed by Liu [28], the convolutional neural network (CNN) based image fusion method proposed by Liu [29], the multi-channel medical image fusion (CT) proposed by Zhu [25], the multi-modality image fusion method with joint patch clustering based dictionary learning (KIM) proposed by Kim [30], the image fusion based on multi-scale transform and sparse representation (MST-SR) proposed by Liu [10], a novel infrared and visual image fusion algorithm based on NSST and improved PCNN (NSST-PCNN) was proposed by Li [31], and an infrared and visible image fusion scheme based on NSCT and PC information (NSCT-PC) proposed by Li [9]. This section only picks the fused results of six comparative experiments from thirty attempts to analyze the fusion performance.…”

Section: Experiments Results Of Infrared-visible Image Fusionmentioning

confidence: 99%

“…Therefore, it is necessary to use multiple metrics to do comprehensive performance analysis. This paper uses five objective metrics to evaluate the performances of different fusion methods, which include Q TE [22,23], Q AB/F [24,25], Q MI [23], Q CB [23,26], and Q V IF [25,27]. Q TE is used to evaluate the Tsallis entropy of the fused image.…”

Section: Objective Evaluation Metricsmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

Huang

Wei

et al. 2019

Entropy

Self Cite

View full text Add to dashboard Cite

In multi-modality image fusion, source image decomposition, such as multi-scale transform (MST), is a necessary step and also widely used. However, when MST is directly used to decompose source images into high-and low-frequency components, the corresponding decomposed components are not precise enough for the following infrared-visible fusion operations. This paper proposes a non-subsampled contourlet transform (NSCT) based decomposition method for image fusion, by which source images are decomposed to obtain corresponding high-and low-frequency sub-bands. Unlike MST, the obtained high-frequency sub-bands have different decomposition layers, and each layer contains different information. In order to obtain a more informative fused high-frequency component, maximum absolute value and pulse coupled neural network (PCNN) fusion rules are applied to different sub-bands of high-frequency components. Activity measures, such as phase congruency (PC), local measure of sharpness change (LSCM), and local signal strength (LSS), are designed to enhance the detailed features of fused low-frequency components. The fused high-and low-frequency components are integrated to form a fused image. The experiment results show that the fused images obtained by the proposed method achieve good performance in clarity, contrast, and image information entropy.Entropy 2019, 21, 1135 2 of 16 for infrared-visible image fusion, have been widely applied to the target recognition in different environments, such as smart city, battlefield, remote sensing, and so on [2,3].In recent years, transform domain based methods have become the mainstream in infrared-visible image fusion, which include pyramid, wavelet transform, multi-scale geometric transform, sparse representation [4,5], and so on. Pyramid, wavelet transform, and multi-scale geometric transform can be categorized as MST-based methods. MST-based methods have three main steps. First, MST is employed to decompose each source image into high-frequency sub-bands at different scales and directions as well as one low-frequency sub-band. Then, the obtained high-and low-frequency sub-bands are fused separately following different fusion rules. Finally, the fused image is obtained by performing the inverse MST on both fused high-and low-frequency sub-bands. Double-tree complex wavelet transform as a kind of wavelet transform can only capture a limited amount of edge information, but cannot correctly and effectively represent the discontinuity of lines and curves [6]. As a true two-dimensional (2D) multi-scale geometric analysis method, contourlet transform (CT) possesses localization, multi-resolution, multi-scale, multi-direction, and anisotropy.As a shift-invariant version of CT, NSCT performs well in transform domain, and has been widely used in image fusion. NSCT has multi-scale and multi-direction features, which can solve the limitations of traditional wavelet methods in the representation of image curves and edges [7]. Compared with traditional MST-based image fusion methods, NSCT has shif...

show abstract

Section: Experiments Results Of Infrared-visible Image Fusionmentioning

confidence: 99%

Section: Objective Evaluation Metricsmentioning

confidence: 99%

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

Huang

Wei

et al. 2019

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…Combination of complex contourlet transform with wavelet can obtain robust image fusion [34]. Transform-based methods can be applied to liver diagnosis [35], prediction of multifactorial diseases [36], parametric classification [36], and multi-modality image fusion [37,38].…”

Section: Image Fusionmentioning

confidence: 99%

An Integrated Dictionary-Learning Entropy-Based Medical Image Fusion Framework

Wang

Zhang

et al. 2017

Future Internet

Self Cite

View full text Add to dashboard Cite

Abstract:Image fusion is widely used in different areas and can integrate complementary and relevant information of source images captured by multiple sensors into a unitary synthetic image. Medical image fusion, as an important image fusion application, can extract the details of multiple images from different imaging modalities and combine them into an image that contains complete and non-redundant information for increasing the accuracy of medical diagnosis and assessment. The quality of the fused image directly affects medical diagnosis and assessment. However, existing solutions have some drawbacks in contrast, sharpness, brightness, blur and details. This paper proposes an integrated dictionary-learning and entropy-based medical image-fusion framework that consists of three steps. First, the input image information is decomposed into low-frequency and high-frequency components by using a Gaussian filter. Second, low-frequency components are fused by weighted average algorithm and high-frequency components are fused by the dictionary-learning based algorithm. In the dictionary-learning process of high-frequency components, an entropy-based algorithm is used for informative blocks selection. Third, the fused low-frequency and high-frequency components are combined to obtain the final fusion results. The results and analyses of comparative experiments demonstrate that the proposed medical image fusion framework has better performance than existing solutions.

show abstract

“…Compared with HSIs, multispectral images (MSI) have wider bandwidth, and are often of higher spatial resolution (e.g., ASTER MSI is of 15 m resolution). Fusing low resolution (LR) HSIs with a high resolution (HR) MSIs is an important technology to enhance the spatial resolution of HSI [4,5].…”

Section: Introductionmentioning

confidence: 99%

“…Several HSI-MSI fusion algorithms have been proposed in the last decades [4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20]. HR HSI can be reconstructed by combining endmember of LR HSI and an abundance of HR MSI.…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

2018

View full text Add to dashboard Cite

Abstract:Enhancing the spatial resolution of hyperspectral image (HSI) is of significance for applications. Fusing HSI with a high resolution (HR) multispectral image (MSI) is an important technology for HSI enhancement. Inspired by the success of deep learning in image enhancement, in this paper, we propose a HSI-MSI fusion method by designing a deep convolutional neural network (CNN) with two branches which are devoted to features of HSI and MSI. In order to exploit spectral correlation and fuse the MSI, we extract the features from the spectrum of each pixel in low resolution HSI, and its corresponding spatial neighborhood in MSI, with the two CNN branches. The extracted features are then concatenated and fed to fully connected (FC) layers, where the information of HSI and MSI could be fully fused. The output of the FC layers is the spectrum of the expected HR HSI. In the experiment, we evaluate the proposed method on Airborne Visible Infrared Imaging Spectrometer (AVIRIS), and Environmental Mapping and Analysis Program (EnMAP) data. We also apply it to real Hyperion-Sentinel data fusion. The results on the simulated and the real data demonstrate that the proposed method is competitive with other state-of-the-art fusion methods.

show abstract

A novel multi-modality image fusion method based on image decomposition and sparse representation

Cited by 329 publications

References 44 publications

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

A Novel Infrared and Visible Image Information Fusion Method Based on Phase Congruency and Image Entropy

An Integrated Dictionary-Learning Entropy-Based Medical Image Fusion Framework

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Contact Info

Product

Resources

About