Multimodal Medical Image Fusion of Positron Emission Tomography and Magnetic Resonance Imaging Using Generative Adversarial Networks

Abstract

Multimodal medical image fusion is a current technique applied in the applications related to medical field to combine images from the same modality or different modalities to improve the visual content of the image to perform further operations like image segmentation. Biomedical research and medical image analysis highly demand medical image fusion to perform higher level of medical analysis. Multimodal medical fusion assists medical practitioners to visualize the internal organs and tissues. Multimodal medical fusion of brain image helps to medical practitioners to simultaneously visualize hard portion like skull and soft portion like tissue. Brain tumor segmentation can be accurately performed by utilizing the image obtained after multimodal medical image fusion. The area of the tumor can be accurately located with the information obtained from both Positron Emission Tomography and Magnetic Resonance Image in a single fused image. This approach increases the accuracy in diagnosing the tumor and reduces the time consumed in diagnosing and locating the tumor. The functional information of the brain is available in the Positron Emission Tomography while the anatomy of the brain tissue is available in the Magnetic Resonance Image. Thus, the spatial characteristics and functional information can be obtained from a single image using a robust multimodal medical image fusion model. The proposed approach uses a generative adversarial network to fuse Positron Emission Tomography and Magnetic Resonance Image into a single image. The results obtained from the proposed approach can be used for further medical analysis to locate the tumor and plan for further surgical procedures. The performance of the GAN based model is evaluated using two metrics, namely, structural similarity index and mutual information. The proposed approach achieved a structural similarity index of 0.8551 and a mutual information of 2.8059.

Towards full-stack deep learning-empowered data processing pipeline for synchrotron tomography experiments

“…In synchrotron tomography, conventional and machine learning methods 120 , 121 are still considered the most popular, while DL approaches from other data-intensive scientific fields are also worth investigating. GAN-based architectures have been found to be effective 122 , 123 in performing registration and correlation from different modalities because the end-to-end generative nature of multimodal tomographic data fusion workload is well suited. In 2022, Abirami et al.…”

“…In 2022, Abirami et al. 122 used GAN to successfully register single-layer slices obtained from magnetic resonance imaging (MRI) and positron emission tomography (PET) scans, achieving excellent contrast on soft tissues using MRI and better spatial characteristics on bones using PET. The advantages of the two modalities are effectively complemented through modality fusion.…”

Intra Fusion Based CNN Technique for MRI Multimodal Brain Tumor Classification and Segmentation

“…To combine these intensities of four modalities and provide more informative MRI brain tumor image, image fusion is introduced in this process. The method of combining different images that will produce single image with good quality is referred as image fusion [3].…”

Retracted: Multimodal Medical Image Fusion of Positron Emission Tomography and Magnetic Resonance Imaging Using Generative Adversarial Networks

“…This article has been retracted by Hindawi following an investigation undertaken by the publisher [1]. This investigation has uncovered evidence of one or more of the following indicators of systematic manipulation of the publication process:…”

“…This article has been retracted by Hindawi following an investigation undertaken by the publisher [ 1 ]. This investigation has uncovered evidence of one or more of the following indicators of systematic manipulation of the publication process: Discrepancies in scope Discrepancies in the description of the research reported Discrepancies between the availability of data and the research described Inappropriate citations Incoherent, meaningless and/or irrelevant content included in the article Manipulated or compromised peer review …”