Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis

Abstract

For the last decade, it has been shown that neuroimaging can be a potential tool for the diagnosis of Alzheimer’s Disease (AD) and its prodromal stage, Mild Cognitive Impairment (MCI), and also fusion of different modalities can further provide the complementary information to enhance diagnostic accuracy. Here, we focus on the problems of both feature representation and fusion of multimodal information from Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET). To our best knowledge, the previous methods in the literature mostly used hand-crafted features such as cortical thickness, gray matter densities from MRI, or voxel intensities from PET, and then combined these multimodal features by simply concatenating into a long vector or transforming into a higher-dimensional kernel space. In this paper, we propose a novel method for a high-level latent and shared feature representation from neuroimaging modalities via deep learning. Specifically, we use Deep Boltzmann Machine (DBM)1, a deep network with a restricted Boltzmann machine as a building block, to find a latent hierarchical feature representation from a 3D patch, and then devise a systematic method for a joint feature representation from the paired patches of MRI and PET with a multimodal DBM. To validate the effectiveness of the proposed method, we performed experiments on ADNI dataset and compared with the state-of-the-art methods. In three binary classification problems of AD vs. healthy Normal Control (NC), MCI vs. NC, and MCI converter vs. MCI non-converter, we obtained the maximal accuracies of 95.35%, 85.67%, and 74.58%, respectively, outperforming the competing methods. By visual inspection of the trained model, we observed that the proposed method could hierarchically discover the complex latent patterns inherent in both MRI and PET.

Lung Cancer Detection and Classification with 3D Convolutional Neural Network (3D-CNN)

“…Suk et al [5] suggested a new latent and shared feature representation of neuro-imaging data of brain using Deep Boltzmann Machine (DBM) for AD/MCI diagnosis. Wu et al [6] developed deep feature learning for deformable registration of brain MR images to improve image registration by using deep features.…”

A Review of Computer Vision Methods for Purpose on Computer-Aided Diagnosis

“…That method showed the exceptional performance compared those of other methods. Suk (25) defined the health state and development model of deep learning that can classify defined health state from multi sensor data. The proposed method has a good performance compared with four existing diagnosis techniques (26).…”

Automated diagnosis of Lungs Tumor Using Segmentation Techniques

“…In Riccardi et al [16] the authors presented a new algorithm to automatically detect nodules with 71% overall accuracy using 3D radical transforms . In neuro imaging data of brain using deep Boltzmann machine for AD/MDC diagnosis is done by the author Suk et al [17]. The method achieves a excellent diagnostic accuracy of 95.52%.To the best of our knowledge there has been no work that uses deep features from lung nodule classification.…”