Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Liu, Yang; Yan, Binyu; Zhang, Rongzhu; Li, Kai; Jeon, Gwanggil; Yang, Xiaoming

doi:10.3390/e24060843

Cited by 8 publications

(5 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The decomposition module decomposes the low-light image into lighting map and reflectance map; the illumination enhancement module is responsible for enhancing the low-light image and denoising the reflectance map before outputting the reconstructed image. In recent years, deep neural networks have been widely used in the field of image enhancement with good results due to their powerful nonlinear fitting ability [16]. Jiang Hai et al [17] proposed a novel Real-low to Real-normal Network (R2RNet) based on Retinex theory.…”

Section: Related Workmentioning

confidence: 99%

“…In Section 3.3.1, the tandem structure of CAM and SAM suffers from distribution lag in feature extraction, which leads to an overall distortion of the color of the enhanced low-light image, whereas connecting the two in parallel focuses on both the light information and the color information of the low-light image. In addition, it was shown in [16] that low-light images have extremely high local dependencies between neighboring pixel points, While PAM can adaptively rescale the per-pixel weights for all input feature mappings. For this reason, we integrated PAM behind the parallel structure of CAM and SAM to adjust all input features channel-by-channel and pixel-by-pixel.…”

Section: Plos Onementioning

confidence: 99%

See 1 more Smart Citation

MTIE-Net: Multi-technology fusion of low-light image enhancement network

Tao,

Wu,

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

Images obtained in low-light scenes are often accompanied by problems such as low visibility, blurred details, and color distortion, enhancing them can effectively improve the visual effect and provide favorable conditions for advanced visual tasks. In this study, we propose a Multi-Technology Fusion of Low-light Image Enhancement Network (MTIE-Net) that modularizes the enhancement task. MTIE-Net consists of a residual dense decomposition network (RDD-Net) based on Retinex theory, an encoder-decoder denoising network (EDD-Net), and a parallel mixed attention-based self-calibrated illumination enhancement network (PCE-Net). The low-light image is first decomposed by RDD-Net into a lighting map and reflectance map; EDD-Net is used to process noise in the reflectance map; Finally, the lighting map is fused with the denoised reflectance map as an input to PCE-Net, using the Fourier transform for illumination enhancement and detail recovery in the frequency domain. Numerous experimental results show that MTIE-Net outperforms the comparison methods in terms of image visual quality enhancement improvement, denoising, and detail recovery. The application in nighttime face detection also fully demonstrates its promise as a pre-processing means in practical applications.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Plos Onementioning

confidence: 99%

MTIE-Net: Multi-technology fusion of low-light image enhancement network

Tao,

Wu,

et al. 2024

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Liu, et al propose a convolutional based CT and MRI image fusion model MMAN (46). The model consists of three parts: two separate encoder blocks for each modality (CT and MRI), a fusion block and a decoder block (Figure 3).…”

Section: Beyond Cardiovascularmentioning

confidence: 99%

“…We included Liu, et al work since they introduce a novel architecture that should be easily adaptable to cardiovascular imaging, and evaluate their model purely on image fusion itself (46). Soenksen, et al provide a framework on how large health record datasets combined with imaging information can be used for a variety of predictive tasks with a simple joining of architectures (51).…”

Section: Beyond Cardiovascularmentioning

confidence: 99%

Applications of AI in multi-modal imaging for cardiovascular disease

Milosevic,

Jin,

Singh

et al. 2024

Front. Radiol.

View full text Add to dashboard Cite

Data for healthcare is diverse and includes many different modalities. Traditional approaches to Artificial Intelligence for cardiovascular disease were typically limited to single modalities. With the proliferation of diverse datasets and new methods in AI, we are now able to integrate different modalities, such as magnetic resonance scans, computerized tomography scans, echocardiography, x-rays, and electronic health records. In this paper, we review research from the last 5 years in applications of AI to multi-modal imaging. There have been many promising results in registration, segmentation, and fusion of different magnetic resonance imaging modalities with each other and computer tomography scans, but there are still many challenges that need to be addressed. Only a few papers have addressed modalities such as x-ray, echocardiography, or non-imaging modalities. As for prediction or classification tasks, there have only been a couple of papers that use multiple modalities in the cardiovascular domain. Furthermore, no models have been implemented or tested in real world cardiovascular clinical settings.

show abstract

“…Small filters are convolved throughout the input image in a sequence of convolutional layers in order to gather regional patterns and characteristics. The boundaries, surfaces, and contours that make up the graphical world are captured by these features [22]. The feature maps are then down sampled by pooling layers, which reduces the computational burden while preserving crucial information.…”

Section: Feature Extraction By Convolutional Neural Network (Cnn)mentioning

confidence: 99%

Using Generative Adversarial Networks and Ensemble Learning for Multi-Modal Medical Image Fusion to Improve the Diagnosis of Rare Neurological Disorders

Reddy,

Rangaswamy,

Bharadwaja

et al. 2023

IJACSA

View full text Add to dashboard Cite

The research suggests a unique ensemble learning approach for precise feature extraction and feature fusion from multi-modal medical pictures, which may be applied to the diagnosis of uncommon neurological illnesses. The proposed method makes use of the combined characteristics of Convolutional Neural Networks and Generative Adversarial Networks (CNN-GAN) to improve diagnostic accuracy and enable early identification. In order to do this, a diverse dataset of multi-modal patient medical records with rare neurological disorders was gathered. The multi-modal pictures are successfully combined using a GAN-based image-to-image translation technique to produce fake images that effectively gather crucial clinical data from different paradigms. To extract features from extensive clinical imaging databases, the research employs trained models using transfer learning approaches with CNN frameworks designed specifically for analyzing medical images. By compiling unique traits from each modality, a thorough grasp of the core pathophysiology is produced. By combining the strengths of several CNN algorithms using ensemble learning techniques including voting by majority, weight averaging, and layering, the forecasts were also integrated to arrive at the final diagnosis. In addition, the ensemble approach enhances the robustness and reliability of the assessment algorithm, resulting in increased effectiveness in identifying unusual neurological conditions. The analysis of the collected data shows that the proposed technique outperforms single-modal designs, demonstrating the importance of multimodal fusion of pictures and feature extraction. The proposed method significantly outperforms existing methods, achieving an accuracy of 99.99%, as opposed to 85.69% for XGBoost and 96.12% for LSTM. The proposed method significantly outperforms existing methods, achieving an average increase in accuracy of approximately 13.3%. The proposed method was implemented using Python software.

show abstract

Multi-Scale Mixed Attention Network for CT and MRI Image Fusion

Cited by 8 publications

References 51 publications

MTIE-Net: Multi-technology fusion of low-light image enhancement network

MTIE-Net: Multi-technology fusion of low-light image enhancement network

Applications of AI in multi-modal imaging for cardiovascular disease

Using Generative Adversarial Networks and Ensemble Learning for Multi-Modal Medical Image Fusion to Improve the Diagnosis of Rare Neurological Disorders

Contact Info

Product

Resources

About