An Effective Multimodal Image Fusion Method Using MRI and PET for Alzheimer's Disease Diagnosis

Abstract

Alzheimer's disease (AD) is an irreversible brain disease that severely damages human thinking and memory. Early diagnosis plays an important part in the prevention and treatment of AD. Neuroimaging-based computer-aided diagnosis (CAD) has shown that deep learning methods using multimodal images are beneficial to guide AD detection. In recent years, many methods based on multimodal feature learning have been proposed to extract and fuse latent representation information from different neuroimaging modalities including magnetic resonance imaging (MRI) and 18-fluorodeoxyglucose positron emission tomography (FDG-PET). However, these methods lack the interpretability required to clearly explain the specific meaning of the extracted information. To make the multimodal fusion process more persuasive, we propose an image fusion method to aid AD diagnosis. Specifically, we fuse the gray matter (GM) tissue area of brain MRI and FDG-PET images by registration and mask coding to obtain a new fused modality called “GM-PET.” The resulting single composite image emphasizes the GM area that is critical for AD diagnosis, while retaining both the contour and metabolic characteristics of the subject's brain tissue. In addition, we use the three-dimensional simple convolutional neural network (3D Simple CNN) and 3D Multi-Scale CNN to evaluate the effectiveness of our image fusion method in binary classification and multi-classification tasks. Experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset indicate that the proposed image fusion method achieves better overall performance than unimodal and feature fusion methods, and that it outperforms state-of-the-art methods for AD diagnosis.

Multi-feature computational framework for combined signatures of dementia in underrepresented settings

“…different markers of cognitive screening, EEG, MRI, ad fMRI) for a combined classification of AD and bvFTD using a technique from machine learning called feature importance analysis. Most of multimodal machine learning approaches for AD characterization [24][25][26][27][28] focusing on MRI data come from high-income countries such as the Alzheimer's Disease Neuroimaging Initiative (ADNI) [29] and Open Access Series of Imaging Studies (OASIS) [30] databases, where both neuroimaging parameters and sample DEMs are homogeneous. Conversely, our work is developed for real-life clinical scenarios with heterogenous acquisition parameters and patients' diversity across SACs.…”

Detection of β-amyloid aggregates/plaques in 5xFAD mice by labelled native PLGA nanoparticles: implication in the diagnosis of Alzheimer’s disease

“…High sensitivity is of interest especially in prodromal phases of AD when plaque load is relatively low [ 9 ]. Interestingly, with the advancements of various imaging techniques, multimodal imaging (i.e., PET/MRI; MRI/SPECT; PET/CT; structural/functional MRI) modalities are gaining momentum in AD diagnosis to obtain precise details and enhanced spatial resolution at specific brain sites for comprehensive assessment [ 37 , 38 ]. Thus, extensive research is currently being carried out to develop new tracers that can not only overcome the limitations of previous agents, i.e., safety, efficacy, non-specific binding and rather short half-lives, but also can assist in emerging therapies as well as in the progression of disease pathology.…”

Automated detection of Alzheimer’s disease: a multi-modal approach with 3D MRI and amyloid PET

“…Huang et al’s work 23 , closely related to our research, employed a VGG-inspired 3D CNN to process MRI and FDG-PET data simultaneously, yielding a remarkable 90% accuracy in distinguishing between healthy subjects and those with AD. Song et al 24 adopted a distinct strategy by creating a synthetic “fused” volume from MRI and PET modalities rather than extracting and combining features separately.…”