Using classification to identify biomarkers for various brain disorders has become a common practice among the functional MR imaging community. Typical classification pipeline includes taking the time series, extracting features from them, and using them to classify a set of patients and healthy controls. The most informative features are then presented as novel biomarkers. In this paper, we compared the results of single and double cross validation schemes on a cohort of 170 subjects with schizophrenia and healthy control subjects. We used graph theoretic measures as our features, comparing the use of functional and anatomical atlases to define nodes and the effect of prewhitening to remove autocorrelation trends. We found that double cross validation resulted in a 20% decrease in classification performance compared to single cross validation. The anatomical atlas resulted in higher classification results. Prewhitening resulted in a 10% boost in classification performance. Overall, a classification performance of 80% was obtained with a double-cross validation scheme using prewhitened time series and an anatomical brain atlas. However, reproducibility of classification within subjects across scans was surprisingly low and comparable to across subject classification rates, indicating that subject state during the short scan significantly influences the estimated features and classification performance.

No abstract

Brain parcellations play a crucial role in the analysis of brain imaging data sets, as they can significantly affect the outcome of the analysis. In recent years, several novel approaches for constructing MRI-based brain parcellations have been developed with promising results. In the absence of ground truth, several evaluation approaches have been used to evaluate currently available brain parcellations. In this article, we review and critique methods used for evaluating functional brain parcellations constructed using fMRI data sets. We also describe how some of these evaluation methods have been used to estimate the optimal parcellation granularity. We provide a critical discussion of the current approach to the problem of identifying the optimal brain parcellation that is suited for a given neuroimaging study. We argue that the criteria for an optimal brain parcellation must depend on the application the parcellation is intended for. We describe a teleological approach to the evaluation of brain parcellations, where brain parcellations are evaluated in different contexts and optimal brain parcellations for each context are identified separately. We conclude by discussing several directions for further research that would result in improved evaluation strategies.