Major depressive disorder (MDD) has been the subject of many neuroimaging case–control classification studies. Although some studies report accuracies ≥80%, most have investigated relatively small samples of clinically‐ascertained, currently symptomatic cases, and did not attempt replication in larger samples. We here first aimed to replicate previously reported classification accuracies in a small, well‐phenotyped community‐based group of current MDD cases with clinical interview‐based diagnoses (from STratifying Resilience and Depression Longitudinally cohort, ‘STRADL’). We performed a set of exploratory predictive classification analyses with measures related to brain morphometry and white matter integrity. We applied three classifier types—SVM, penalised logistic regression or decision tree—either with or without optimisation, and with or without feature selection. We then determined whether similar accuracies could be replicated in a larger independent population‐based sample with self‐reported current depression (UK Biobank cohort). Additional analyses extended to lifetime MDD diagnoses—remitted MDD in STRADL, and lifetime‐experienced MDD in UK Biobank. The highest cross‐validation accuracy (75%) was achieved in the initial current MDD sample with a decision tree classifier and cortical surface area features. The most frequently selected decision tree split variables included surface areas of bilateral caudal anterior cingulate, left lingual gyrus, left superior frontal, right precentral and paracentral regions. High accuracy was not achieved in the larger samples with self‐reported current depression (53.73%), with remitted MDD (57.48%), or with lifetime‐experienced MDD (52.68–60.29%). Our results indicate that high predictive classification accuracies may not immediately translate to larger samples with broader criteria for depression, and may not be robust across different classification approaches.

With several initiatives well underway towards amassing large and high-quality population-based neuroimaging datasets, deep learning is set to push the boundaries of what is possible in classification and prediction in neuroimaging studies. This includes those that derive increasingly popular structural connectomes, which map out the connections (and their relative strengths) between brain regions. Here, we test different Convolutional Neural Network (CNN) models in a benchmark sex prediction task in a large sample of N=3,152 structural connectomes acquired from the UK Biobank, and compare results across different connectome processing choices. The best results (76.5% test accuracy) were achieved using Fractional Anisotropy (FA) weighted connectomes, without sparsification, and with a simple weight normalisation through division by the maximum FA value. We also confirm that for structural connectomes, a Graph CNN approach, the recently proposed BrainNetCNN, outperforms an image-based CNN.

There is an increasing expectation that advanced, computationally expensive machine learning (ML) techniques, when applied to large population‐wide neuroimaging datasets, will help to uncover key differences in the human brain in health and disease. We take a comprehensive approach to explore how multiple aspects of brain structural connectivity can predict sex, age, general cognitive function and general psychopathology, testing different ML algorithms from deep learning (DL) model (BrainNetCNN) to classical ML methods. We modelled N = 8183 structural connectomes from UK Biobank using six different structural network weightings obtained from diffusion MRI. Streamline count generally provided the highest prediction accuracies in all prediction tasks. DL did not improve on prediction accuracies from simpler linear models. Further, high correlations between gradient attribution coefficients from DL and model coefficients from linear models suggested the models ranked the importance of features in similar ways, which indirectly suggested the similarity in models' strategies for making predictive decision to some extent. This highlights that model complexity is unlikely to improve detection of associations between structural connectomes and complex phenotypes with the current sample size.

There is increasing expectation that advanced, computationally expensive machine learning techniques, when applied to large population-wide neuroimaging datasets, will help to uncover key differences in the human brain in health and disease. We take a comprehensive approach to explore how multiple aspects of brain structural connectivity can predict sex, age, general cognitive function and general psychopathology, testing different machine learning algorithms from deep learning model (BrainNetCNN) to classical machine learning methods. We modelled N = 8, 183 structural connectomes from UK Biobank using six different structural network weightings obtained from diffusion MRI. Streamline count generally provided highest prediction accuracies in all prediction tasks. Deep learning did not improve on prediction accuracies from simpler linear models. Further, high correlations between model coefficients from deep learning and linear models suggested similar decision mechanisms were used. This highlights that model complexity is unlikely to improve detection of associations between structural connectomes and complex phenotypes.