Multimodal Medical Image Fusion Framework Based on Simplified PCNN in Nonsubsampled Contourlet Transform Domain

Abstract

In this paper, we present a new medical image fusion algorithm based on nonsubsampled contourlet transform (NSCT) and spiking cortical model (SCM). The flexible multi-resolution, anisotropy, and directional expansion characteristics of NSCT are associated with global coupling and pulse synchronization features of SCM. Considering the human visual system characteristics, two different fusion rules are used to fuse the low and high frequency sub-bands respectively. Firstly, maximum selection rule (MSR) is used to fuse low frequency coefficients. Secondly, spatial frequency (SF) is applied to motivate SCM network rather than using coefficients value directly, and then the time matrix of SCM is set as criteria to select coefficients of high frequency subband. The effectiveness of the proposed algorithm is achieved by the comparison with existing fusion methods

Review of Image Fusion Based on Pulse-Coupled Neural Network

“…In their paper, besides row and column frequency, diagonal frequency is added for improving the ability of capturing directional information. Wang et al [52] also adopt this framework, however, they try to employ spiking cortical model, dual-channel PCNN [53] and improve PCNN [54] to fuse HF coefficients, respectively. In these papers, the maximum selection rule (MSR) is used to fuse LF coefficients while spatial frequency of high frequency subband is considered as the gradient features of images to motivate neural networks.…”

An overview of multi-modal medical image fusion

“…The corresponding pixels of the two input images have been already registered. In this section, the methods for comparison are six image fusion algorithms based on IHS+PCA method [3], Morphology Pyramid (MoR)+MR method [5], DWT+Entropy [7], NSCT+PCNN [37], Joint sparse representation(JSP)+AR [20] and LES+DC [24] fusion methods, respectively. To evaluate the performance of different multi-modal medical image fusion methods described in Section II and Section III, eight metrics are adopted as the objective quality assessments, such as SSIM in Eq.…”

“…Inspired by the HVS, the algorithms for detecting the corners, edges and saliency features are used as fusion rules in multi-modal medical image fusion, such as visibility [30], smallest univalve segment assimilating nucleus (SUSAN) [33], artificial neural networks (ANN) [34][35][36][37] and retina-inspired model (RIM) [4,38,39].…”

“…Examples of ANN models are mapping neural network (MNN)[34] and pulse coupled neural network (PCNN)[35][36][37].MNN model is inspired from the self-organizing neural network and provides multi-level fusion strategies. In addition, PCNN model has been widely used as the image fusion rules for low and high frequency bands decomposed by WT methods.…”

“…model[34][35][36][37] is able to learn from inputs for processing features. Examples of ANN models are mapping neural network (MNN)[34] and pulse coupled neural network (PCNN)[35][36][37].MNN model is inspired from the self-organizing neural network and provides multi-level fusion strategies.…”