3D-SiameseNet to Analyze Brain MRI

Abstract

Prediction of the cognitive evolution of a person susceptible to develop a neurodegenerative disorder is crucial to provide an appropriate treatment as soon as possible. In this paper we propose a 3D siamese network designed to extract features from whole-brain 3D MRI images. We show that it is possible to extract meaningful features using convolution layers, reducing the need of classical image processing operations such as segmentation or pre-computing features such as cortical thickness. To lead this study we used the Alzheimer's Disease Neuroimaging Initiative (ADNI), a public data base of 3D MRI brain images. A set of 247 subjects has been extracted, all of the subjects having 2 images in a range of 12 months. In order to measure the evolution of the patients states we have compared these 2 images. Our work has been inspired at the beginning by an article of Bhagwat et al. in 2018, who have proposed a siamese network to predict the status of patients but without any convolutional layers and reducing the MRI images to a vector of features extracted from predefined ROIs. We show that our network achieves an accuracy of 90% in the classification of cognitively declining VS stable patients. This result has been obtained without the help of a cognitive score and with a small number of patients comparing to the current datasets size claimed in deep learning domain.

Deep learning prediction of mild cognitive impairment conversion to Alzheimer’s disease at 3 years after diagnosis using longitudinal and whole-brain 3D MRI

“…The use of clinical variables could have contributed substantially to higher accuracy. Ostertag et al used a similar Siamese network but employed whole-brain MRI to predict decline in patients’ MMSE score, yielding a validation accuracy of 0.90, but no independent evaluation on a separate test dataset was performed ( Ostertag, Beurton-Aimar & Urruty, 2019 ). Moreover, these two studies differed from ours in that they mixed AD, NC, and MCI patients together, and thus their prediction accuracies are not directly comparable to those from MCI to AD conversion studies because the baseline diagnosis of NC or AD by itself is a strong predictor of neurocognitive decline.…”

“…The use of a Siamese network architecture to analyze longitudinal changes in disease progression from medical images was explored by Li et al and specifically studied in AD brain MRIs by Bhagwat et al and Ostertag et al ( Bhagwat et al, 2018 ; Ostertag, Beurton-Aimar & Urruty, 2019 ; Li et al, 2020 ). The idea behind Siamese networks is that both images are processed by the convolutional layers with identical parameters, with equivalent flattened sets of features for each image at the end of the convolutions.…”

“…Bhagwat used a neural network (albeit not deep learning) and extracted cortical thicknesses from MRIs at two time points to predict decline in Mini-Mental Status Exam (MMSE) scores ( Bhagwat et al, 2018 ). Ostertag et al used a CNN model on whole-brain MRI at two time points to predict decline in MMSE score but did not test their model on an independent testing dataset ( Ostertag, Beurton-Aimar & Urruty, 2019 ). These two studies mixed NC, MCI and AD participants and thus accuracies are not applicable to prediction of MCI to AD conversion.…”

Predicting Brain Degeneration with a Multimodal Siamese Neural Network

“…The MRI sub-module is a slightly modified version of our previous architecture, 3D-SiameseNet [18] (see Fig. 1), a Siamese network in which each branch is made of 3D convolution layers followed by average pooling.…”

Infant brain segmentation based on a combination of VGG‐16 and U‐Net deep neural networks

“…Therefore, in light of the segmentation process, slight changes in the brightness levels of the pixels are extracted from the texture. Segmentation of a special object is based on identifying all pixels in two‐dimensional (2D) images or voxels in 3D images [4, 5].…”