Bootstrapping promotes the RSFC-behavior associations: an application of individual cognitive traits prediction

Abstract

Resting state functional connectivity records enormous functional interaction information between any pair of brain nodes, which enriches the prediction of individual phenotypes. To reduce the high dimensional features in prediction, correlation analysis is a common way for feature selection. However, rs-fMRI signal exhibits typically low signal-to-noise ratio and correlation analysis is sensitive to outliers and data distribution, which may bring unstable and uninformative features to subsequent prediction. To alleviate this problem, a bootstrapping-based feature selection framework was proposed and applied on three widely used regression models: connectome-based predictive model (CPM), support vector regression (SVR) and least absolute shrinkage and selection operator (LASSO). A large open-source dataset from Human Connectome Project (HCP) was adopted in the study and a series of cognitive traits were acted as the prediction targets. To systematically investigate the influences of different parameter settings on the bootstrapping-based framework, a total of 216 parameter combinations were evaluated through the R value between the predicted and real cognitive traits, and the best identified performance among them was chosen out as the final prediction accuracy for each cognitive trait. By using bootstrapping without replacement, the best performances of CPM with positive and negative feature sets, SVR and LASSO averagely increased by 28.0%, 33.2%, 11.6% and 24.3% in R values in contrast to the baseline method without bootstrapping. By using bootstrapping with replacement, these best performances increased by 22.1%, 22.9%, 9.4% and 19.6%. Furthermore, the bootstrapping-based feature selection methods could effectively refine the original feature sets obtained from correlation analysis, which thus retained the more stable and informative feature sets. The results demonstrate that bootstrapping-based feature selection is an easy-to-use and effective method to improve RSFC prediction of cognitive traits and is highly recommended in future RSFC prediction studies.

Resample aggregating improves the generalizability of connectome predictive modeling

“…Given that with the 100 bootstraps, almost 50% of the features occur at least once in the bagged protocol shown here, and indeed 20% of features with subsample 200 and 11% of features with subsample 300, resample aggregation may provide an improvement on identifying noisy features compared to 10-fold cross validation, as well as provide more insight into feature variation. A previous study by Wei et al using bagged models for prediction has also recommended ~100 bootstraps ( Wei et al, 2020 ). An additional factor to note is that the greater the fraction of subjects from the training set that are included in the feature selection step, the less likely the features are to vary across folds.…”

“…Bagging has been applied to fMRI data before to determine brain state ( Richiardi et al, 2011 ), and for brain parcellation (to define an atlas) ( Bellec et al, 2010 ; Nikolaidis et al, 2020 ). With respect to predictive modeling, it been shown to boost within-sample performance of resting state FC based brain-behavior regression models ( Wei et al, 2020 ), and has also been applied in a classification approach ( Hoyos-Idrobo et al, 2018 ).…”