Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation

Liu, Shuaiqi; Wang, Mingwang; Yin, Lu; Sun, Xiuming; Zhang, Yudong; Zhao, Jie

doi:10.3389/fncom.2021.803724

Cited by 16 publications

(5 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SPECT and PET images can provide biological activity and metabolic information of cells at a lower resolution. 8,9 The same is true for a widely used morphological change tracker, GFP. 10 Due to differences in imaging principles and imaging equipment, single-modality molecular images often cannot provide enough information to assist the diagnosis of diseases, whereas multimodal molecular image fusion can bring a more comprehensive understanding and make up for the deficiency.…”

Section: Introductionmentioning

confidence: 82%

“…For a more accurate diagnosis, clinicians or researchers need to perform complementary analysis with the help of functional images, including positron emission tomography (PET), single photon emission computed tomography (SPECT), and green fluorescent protein (GFP). SPECT and PET images can provide biological activity and metabolic information of cells at a lower resolution 8,9 . The same is true for a widely used morphological change tracker, GFP 10 .…”

Section: Introductionmentioning

confidence: 93%

See 1 more Smart Citation

A multimodal molecular image fusion method based on relative total variation and co‐saliency detection

Wang

Zhang

Zhu

2022

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Image fusion can integrate complementary information from multimodal molecular images to provide an informative single result image. In order to obtain a better fusion effect, this article proposes a novel method based on relative total variation and co‐saliency detection (RTVCSD). First, only the gray‐scale anatomical image is decomposed into a base layer and a texture layer according to the relative total variation; then, the three‐channel color functional image is transformed into the luminance and chroma (YUV) color space, and the luminance component Y is directly fused with the base layer of the anatomical image by comparing the co‐saliency information; next, the fused base layer is linearly combined with the texture layer, and the obtained fused result is combined with the chroma information U and V of the functional image. Finally, the fused image is obtained by transforming back to the red–green–blue color space. The dataset consists of magnetic resonance imaging (MRI)/positron emission tomography images, MRI/single photon emission computed tomography (SPECT) images, computed tomography/SPECT images, and green fluorescent protein/phase contrast images, each category with 20 image pairs. Experimental results demonstrate that the proposed method RTVCSD outperforms the nine comparison algorithms in terms of visual effects and objective evaluation. RTVCSD well preserves the texture information of the anatomical image and the metabolism or protein distribution information of the functional image.

show abstract

Section: Introductionmentioning

confidence: 82%

Section: Introductionmentioning

confidence: 93%

A multimodal molecular image fusion method based on relative total variation and co‐saliency detection

Wang

Zhang

Zhu

2022

Int J Imaging Syst Tech

View full text Add to dashboard Cite

show abstract

“…Shift invariance is the most desirable property and is applied in various applications of image processing. These are: image watermarking [22], image enhancement [23], image fusion [24], and image deblurring [25]. The above mentioned drawbacks are addressed by the non-subsampled contourlet transform (NSCT) [26] and non-subsampled Shearlet transform (NSST) [27,28].…”

Section: Related Workmentioning

confidence: 99%

An Improved Multimodal Medical Image Fusion Approach Using Intuitionistic Fuzzy Set and Intuitionistic Fuzzy Cross-Correlation

Haribabu

Guruviah

2023

Diagnostics

View full text Add to dashboard Cite

Multimodal medical image fusion (MMIF) is the process of merging different modalities of medical images into a single output image (fused image) with a significant quantity of information to improve clinical applicability. It enables a better diagnosis and makes the diagnostic process easier. In medical image fusion (MIF), an intuitionistic fuzzy set (IFS) plays a role in enhancing the quality of the image, which is useful for medical diagnosis. In this article, a new approach to intuitionistic fuzzy set-based MMIF has been proposed. Initially, the input medical images are fuzzified and then create intuitionistic fuzzy images (IFIs). Intuitionistic fuzzy entropy plays a major role in calculating the optimal value for three degrees, namely, membership, non-membership, and hesitation. After that, the IFIs are decomposed into small blocks and then perform the fusion rule. Finally, the enhanced fused image can be obtained by the defuzzification process. The proposed method is tested on various medical image datasets in terms of subjective and objective analysis. The proposed algorithm provides a better-quality fused image and is superior to other existing methods such as PCA, DWTPCA, contourlet transform (CONT), DWT with fuzzy logic, Sugeno’s intuitionistic fuzzy set, Chaira’s intuitionistic fuzzy set, and PC-NSCT. The assessment of the fused image is evaluated with various performance metrics such as average pixel intensity (API), standard deviation (SD), average gradient (AG), spatial frequency (SF), modified spatial frequency (MSF), cross-correlation (CC), mutual information (MI), and fusion symmetry (FS).

show abstract

“…Extensive research has been carried out in recent years on the topic of multimodal image fusion. According to the image transformation strategy adopted in Li et al (2016), fusion categories are mainly concentrated on four aspects: MST-based schemes (Tannaz et al 2020, Goyal et al 2021, Singh et al 2022, sparse representation (SR)based schemes (Yin 2018, Shibu and Priyadharsini 2021, Zhang 2021, deep learning (DL)-based schemes (Vasanthi et al 2021, Liu et al 2022, and the combination of different transformations (Maqsood and Javed 2020, Vanitha et al 2020, Reddy et al 2021. It is worth noting that the applications of SR in voxel selection, compressed sensing, and bioelectrical signal detection prove that SR has become an effective tool for image processing.…”

Section: Introductionmentioning

confidence: 99%

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

Hu,

Cai,

et al. 2024

Phys. Med. Biol.

View full text Add to dashboard Cite

ObjectiveMultimodal medical image fusion (MMIF) technologies merges diverse medical images with rich information, boosting diagnostic efficiency and accuracy. Due to global optimization and single-valued nature, convolutional sparse representation (CSR) outshines the standard sparse representation (SR) in significance. By addressing the challenges of sensitivity to highly redundant dictionaries and robustness to misregistration, an adaptive convolutional sparsity scheme with measurement of the sub-band correlation in the non-subsampled contourlet transform (NSCT) domain is proposed for MMIF.ApproachThe fusion scheme incorporates four main components: image decomposition into two scales, fusion of detail layers, fusion of base layers, and reconstruction of the two scales. We solved a Tikhonov regularization optimization problem with source images to obtain the base and detail layers. Then, after CSR processing, detail layers were sparsely decomposed using pre-trained dictionary filters for initial coefficient maps. NSCT domain's sub-band correlation was used to refine fusion coefficient maps, and sparse reconstruction produced the fused detail layer. Meanwhile, base layers were fused using averaging. The final fused image was obtained via two-scale reconstruction.Main resultsExperimental validation of clinical image sets revealed that the proposed fusion scheme can not only effectively eliminate the interference of partial misregistration, but also outperform the representative state-of-the-art fusion schemes in the preservation of structural and textural details according to subjective visual evaluations and objective quality evaluations.SignificanceThe proposed fusion scheme is competitive due to its low-redundancy dictionary, robustness to misregistration, and better fusion performance. This is achieved by training the dictionary with minimal samples through CSR to adaptively preserve overcompleteness for detail layers, and constructing fusion activity level with sub-band correlation in the NSCT domain to maintain CSR attributes. Additionally, ordering the NSCT for reverse sparse representation further enhances sub-band correlation to promote the preservation of structural and textural details.

show abstract

Two-Scale Multimodal Medical Image Fusion Based on Structure Preservation

Cited by 16 publications

References 53 publications

A multimodal molecular image fusion method based on relative total variation and co‐saliency detection

A multimodal molecular image fusion method based on relative total variation and co‐saliency detection

An Improved Multimodal Medical Image Fusion Approach Using Intuitionistic Fuzzy Set and Intuitionistic Fuzzy Cross-Correlation

Adaptive convolutional sparsity with sub-band correlation in the NSCT domain for MRI image fusion

Contact Info

Product

Resources

About