Multimodal medical image fusion algorithm in the era of big data

Abstract

In image-based medical decision-making, different modalities of medical images of a given organ of a patient are captured. Each of these images will represent a modality that will render the examined organ differently, leading to different observations of a given phenomenon (such as stroke). The accurate analysis of each of these modalities promotes the detection of more appropriate medical decisions. Multimodal medical imaging is a research field that consists in the development of robust algorithms that can enable the fusion of image information acquired by different sets of modalities. In this paper, a novel multimodal medical image fusion algorithm is proposed for a wide range of medical diagnostic problems. It is based on the application of a boundary measured pulse-coupled neural network fusion strategy and an energy attribute fusion strategy in a non-subsampled shearlet transform domain. Our algorithm was validated in dataset with modalities of several diseases, namely glioma, Alzheimer's, and metastatic bronchogenic carcinoma, which contain more than 100 image pairs. Qualitative and quantitative evaluation verifies that the proposed algorithm outperforms most of the current algorithms, providing important ideas for medical diagnosis.

Diagnosis of breast cancer based on modern mammography using hybrid transfer learning

“…Therefore, we can use deep learning for the detection of diseases from image data. Conversely, the image fusion algorithm was conducted for medical images where the big data was efficiently utilized (Tan et al 2020). Traditional deep learning techniques are used for detecting blood cells.…”

NSST domain CT–MR neurological image fusion using optimised biologically inspired neural network

“…Later on, NSSTs are widely explored because of the ability to offer anisotropic, multi-dimensional, multi-directional, and shift-invariant representation of the images with lower computational complexity [13,19,20]. Due to the several advantages, several researchers moved towards the NSST for image fusion giving improved fusion performances with the better preservation of structure detail information [15,16,[21][22][23].…”

“…13 and 14, it is observed that Liu et al [34] and the proposed methods can preserve more structural details but the colour information is better highlighted by the proposed method. Yin et al [31] and the [34], Yin et al [31], Li and Zhao [14], Tan et al [23], (left to right: second row) Panigrahy et al [32], Li et al [7], Proposed method…”

“…Moreover, the proposed method ranks first for Q Y metric and gives 16.16% higher values than the method proposed by Li and Zhao [14], which indicates that the proposed method is best at integrating the complementary information present in source MR-SPECT images. Therefore, it can be concluded based on the experimental results that the proposed fusion method provides [34], Yin et al [31], Li and Zhao [14], Tan et al [23], (left to right: second row) Panigrahy et al [32], Li et al [7], Proposed method significantly better-fused images in both the terms of visual and quantitative performance for MR-SPECT image fusion.…”

“…Performance evaluation of existing fusion approaches with the proposed method for MR-SPECT pair-2 shown in Fig. 14 Performance metricFusion methods Liu et al[34] Yin et al[31] Li and Zhao[14] Tan et al[23] Panigrahy et al[32] Li et al[7] Proposed method…”