Multimodal medical image fusion via laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy

Fu, Jun; Li, Weisheng; Du, Jiao; Xiao, Bin

doi:10.1016/j.compbiomed.2020.104048

Cited by 50 publications

(20 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Multi-modal fusion is used in applications such as audiovisual speech recognition (AVSR) [ 24 ], multi-modal emotion recognition [ 25 , 26 ], and medical-aided diagnosis [ 27 ].…”

Section: Related Workmentioning

confidence: 99%

Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy

Hao

Liu

et al. 2023

Life

View full text Add to dashboard Cite

Membranous nephropathy is one of the most prevalent conditions responsible for nephrotic syndrome in adults. It is clinically nonspecific and mainly diagnosed by kidney biopsy pathology, with three prevalent techniques: light microscopy, electron microscopy, and immunofluorescence microscopy. Manual observation of glomeruli one by one under the microscope is very time-consuming, and there are certain differences in the observation results between physicians. This study makes use of whole-slide images scanned by a light microscope as well as immunofluorescence images to classify patients with membranous nephropathy. The framework mainly includes a glomerular segmentation module, a confidence coefficient extraction module, and a multi-modal fusion module. This framework first identifies and segments the glomerulus from whole-slide images and immunofluorescence images, and then a glomerular classifier is trained to extract the features of each glomerulus. The results are then combined to produce the final diagnosis. The results of the experiments show that the F1-score of image classification results obtained by combining two kinds of features, which can reach 97.32%, is higher than those obtained by using only light-microscopy-observed images or immunofluorescent images, which reach 92.76% and 93.20%, respectively. Experiments demonstrate that considering both WSIs and immunofluorescence images is effective in improving the diagnosis of membranous nephropathy.

show abstract

“…Multi-modal fusion is used in applications such as audiovisual speech recognition (AVSR) [ 24 ], multi-modal emotion recognition [ 25 , 26 ], and medical-aided diagnosis [ 27 ].…”

Section: Related Workmentioning

confidence: 99%

Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy

Hao

Liu

et al. 2023

Life

View full text Add to dashboard Cite

show abstract

“…, etc. [28], [39]- [50]. Table 2 introduces the applications of image fusion between different imaging modalities in the medical field on the human body that mix between mutually functional and anatomical imaging for additional informative and qualitative images [71]- [84].…”

Section: Medical Imaging Modalitiesmentioning

confidence: 99%

“…Recent works for medical imaging fusion using deeplearning techniques have been introduced in [9], [11], [13], [23], [31], [39], [41], [69], [85]- [88].…”

Section: Introductionmentioning

confidence: 99%

A Comprehensive Survey Analysis for Present Solutions of Medical Image Fusion and Future Directions

et al. 2021

View full text Add to dashboard Cite

The track of medical imaging has witnessed several advancements in the last years. Several medical imaging modalities have appeared in the last decades including X-ray, Computed Tomography (CT), Magnetic Resonance (MR), Positron Emission Tomography (PET), Single-Photon Emission Computed Tomography (SPECT) and ultrasound imaging. Generally, medical images are used for the diagnosis purpose. Each type of acquired images has some merits and limitations. To maximize medical images utilization for the purpose of diagnosis, medical imaging fusion trend has appeared as a hot research field. Different medical imaging modalities are fused to obtain new images with complementary information. This paper presents a survey study of medical imaging modalities and their characteristics. In addition, different medical image fusion approaches and their appropriate quality metrics are presented. The main aim of this comprehensive survey analysis is to contribute in the advancement of medical image approaches that can help for better diagnosis of different diseases.INDEX TERMS Medical imaging, fusion process, imaging modalities, dual-tree complex wavelet transform, curvelet transform, discrete wavelet transform, principal component analysis.

show abstract

“…Differential evolution is used to enhance feature selection (Kaur and Singh, 2020). Fu et al (2020) proposed a multimodal medical image fusion method using Laplacian pyramid combined with CNN. Wang et al (2021) proposed a visible and infrared image fusion method based on Laplacian pyramid and GAN.…”

Section: Introductionmentioning

confidence: 99%

Infrared and visible image fusion based on particle swarm optimization and dense block

2022

View full text Add to dashboard Cite

Infrared and visible image fusion aims to preserve essential thermal information and crucial visible details from two types of input images to generate an informative fusion image for better visual perception. In recent years, several hybrid methods have been applied in the field of infrared and visible image fusion. In this paper, we proposed a novel image fusion method based on particle swarm optimization and dense block for visible and infrared images. Particle swarm optimization is utilized to optimize the weighting factors of the coefficients obtained by discrete wavelet transform, then the coefficients are fused with the optimum weight to obtain the initial fusion image. The final fusion image is created by integrating the first fused image with the input visible image using a deep learning model, in which dense block is utilized for better feature extraction ability. The results of comparison experiments demonstrate that our method produces fusion images with richer details and texture features, and the fused image reduces the artifacts and noise.

show abstract

Multimodal medical image fusion via laplacian pyramid and convolutional neural network reconstruction with local gradient energy strategy

Cited by 50 publications

References 31 publications

Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy

Accurate Kidney Pathological Image Classification Method Based on Deep Learning and Multi-Modal Fusion Method with Application to Membranous Nephropathy

A Comprehensive Survey Analysis for Present Solutions of Medical Image Fusion and Future Directions

Infrared and visible image fusion based on particle swarm optimization and dense block

Contact Info

Product

Resources

About