In recent years, 2D convolutional neural networks (CNNs) have been extensively used to diagnose neurological diseases from magnetic resonance imaging (MRI) data due to their potential to discern subtle and intricate patterns. Despite the high performances reported in numerous studies, developing CNN models with good generalization abilities is still a challenging task due to possible data leakage introduced during cross-validation (CV). In this study, we quantitatively assessed the effect of a data leakage caused by 3D MRI data splitting based on a 2D slice-level using three 2D CNN models to classify patients with Alzheimer’s disease (AD) and Parkinson’s disease (PD). Our experiments showed that slice-level CV erroneously boosted the average slice level accuracy on the test set by 30% on Open Access Series of Imaging Studies (OASIS), 29% on Alzheimer’s Disease Neuroimaging Initiative (ADNI), 48% on Parkinson’s Progression Markers Initiative (PPMI) and 55% on a local de-novo PD Versilia dataset. Further tests on a randomly labeled OASIS-derived dataset produced about 96% of (erroneous) accuracy (slice-level split) and 50% accuracy (subject-level split), as expected from a randomized experiment. Overall, the extent of the effect of an erroneous slice-based CV is severe, especially for small datasets.

Alzheimer's Disease (AD) is a widespread neurodegenerative disease caused by structural changes in the brain and leads to deterioration of cognitive functions. Patients usually experience diagnostic symptoms at later stages after irreversible neural damage occurs. Therefore, early detection of AD is crucial to start treatments to decelerate the progress of the disease and to maximize patients' quality of life. With the rapid advances in machine learning and scanning, early detection of AD may be possible via computer-assisted systems using neuroimaging data. Among all, deep learning utilizing magnetic resonance imaging (MRI) has become a prominent tool due to its capability to extract high-level features through local connectivity, weight sharing, and spatial invariance. This paper describes our investigation of the classification accuracy based on two publicly available data sets, namely, ADNI and OASIS, by building a 3D VGG variant convolutional network (CNN). We used 3D models to avoid information loss, which occurs during the process of slicing 3D MRI into 2D images and analyzing them by 2D convolutional filters. We also conducted a pre-processing of the data to enhance the effectiveness and classification performance of the model. The proposed model achieved 73.4% classification accuracy on ADNI and 69.9% on OASIS dataset with 5-fold cross-validation (CV), outperforming 2D network models.