Pairwise feature-based generative adversarial network for incomplete multi-modal Alzheimer’s disease diagnosis

Ye, Haizhou; Zhu, Qi; Yao, Yuan; Jin, Yichao; Zhang, Daoqiang

doi:10.1007/s00371-021-02354-5

Cited by 12 publications

(3 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While most studies have registered subject specific images to a template space ( 19 , 20 , 28 ), only a few have applied methods in native space ( 10 , 29 ). To our knowledge, the choice of using template space versus native space images as input to the GAN has not been evaluated, nor has a justification been provided for the choices made in the disparate studies.…”

Section: Discussionmentioning

confidence: 99%

“…Careful review of previous studies highlights the fact that only increasing the number of training datasets in the machine learning model does not result in better image synthesis quality ( 17 , 19 , 28 ). A lack of data for training and testing of the GAN is unlikely to be a limitation of this study, as the large number of test and training image slices provides adequate variability for robust learning.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Image synthesis of interictal SPECT from MRI and PET using machine learning

Fard,

Reutens,

Ramsay

et al. 2024

Front. Neurol.

View full text Add to dashboard Cite

BackgroundCross-modality image estimation can be performed using generative adversarial networks (GANs). To date, SPECT image estimation from another medical imaging modality using this technique has not been considered. We evaluate the estimation of SPECT from MRI and PET, and additionally assess the necessity for cross-modality image registration for GAN training.MethodsWe estimated interictal SPECT from PET and MRI as a single-channel input, and as a multi-channel input to the GAN. We collected data from 48 individuals with epilepsy and converted them to 3D isotropic images for consistence across the modalities. Training and testing data were prepared in native and template spaces. The Pix2pix framework within the GAN network was adopted. We evaluated the addition of the structural similarity index metric to the loss function in the GAN implementation. Root-mean-square error, structural similarity index, and peak signal-to-noise ratio were used to assess how well SPECT images were able to be synthesised.ResultsHigh quality SPECT images could be synthesised in each case. On average, the use of native space images resulted in a 5.4% percentage improvement in SSIM than the use of images registered to template space. The addition of structural similarity index metric to the GAN loss function did not result in improved synthetic SPECT images. Using PET in either the single channel or dual channel implementation led to the best results, however MRI could produce SPECT images close in quality.ConclusionSynthesis of SPECT from MRI or PET can potentially reduce the number of scans needed for epilepsy patient evaluation and reduce patient exposure to radiation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Image synthesis of interictal SPECT from MRI and PET using machine learning

Fard,

Reutens,

Ramsay

et al. 2024

Front. Neurol.

View full text Add to dashboard Cite

show abstract

“…A framework was proposed for AD diagnosis that utilizes a latent representation space, where complete multi-modality data forms a common representation and incomplete data informs modalityspecific representations (Zhou et al, 2019). A pairwise feature-based generation adversarial network was introduced that leverages MRI features to generate corresponding PET features, reinforced by real PET constraints, and incorporates an attention mechanism to retain structural integrity (Ye et al, 2023). However, these existing methods used pre-defined extracted features from images.…”

Section: Introductionmentioning

confidence: 99%

A Cross-Modal Mutual Knowledge Distillation Framework for Alzheimer’s Disease Diagnosis: Addressing Incomplete Modalities

Kwak,

Mao,

Zheng

et al. 2023

Preprint

View full text Add to dashboard Cite

Early detection of Alzheimer's Disease (AD) is crucial to ensure timely interventions and optimize treatment outcomes for patients. While integrating multi-modal neuroimages, such as MRI and PET, has shown great promise, limited research has been done to effectively handle incomplete multi-modal image datasets in the integration. To this end, we propose a deep learning-based framework that employs Mutual Knowledge Distillation (MKD) to jointly model different sub-cohorts based on their respective available image modalities. In MKD, the model with more modalities (e.g., MRI and PET) is considered a teacher while the model with fewer modalities (e.g., only MRI) is considered a student. Our proposed MKD framework includes three key components: First, we design a teacher model that is student-oriented, namely the Student-oriented Multi-modal Teacher (SMT), through multi-modal information disentanglement. Second, we train the student model by not only minimizing its classification errors but also learning from the SMT teacher. Third, we update the teacher model by transfer learning from the student's feature extractor because the student model is trained with more samples. Evaluations on Alzheimer's Disease Neuroimaging Initiative (ADNI) datasets highlight the effectiveness of our method. Our work demonstrates the potential of using AI for addressing the challenges of incomplete multi-modal neuroimage datasets, opening new avenues for advancing early AD detection and treatment strategies.

show abstract