Integrating across neuroimaging modalities boosts prediction accuracy of cognitive ability

Abstract

Variation in cognitive ability arises from subtle differences in underlying neural architectural properties. Understanding and predicting individual variability in cognition from the differences in brain networks requires harnessing the unique variance captured by different neuroimaging modalities. Here we adopted a multi-level machine learning approach that combines diffusion, functional, and structural MRI data from the Human Connectome Project (N=1050) to provide unitary prediction models of various cognitive abilities: global cognitive function, fluid intelligence, crystallized intelligence, impulsivity, spatial orientation, verbal episodic memory and sustained attention. Out-of-sample predictions of each cognitive score were first generated using a sparsity-constrained principal component regression on individual neuroimaging modalities. These individual predictions were then aggregated and submitted to a LASSO estimator that removed redundant variability across channels. This stacked prediction led to a significant improvement in accuracy, relative to the best single modality predictions (approximately 1% to 4% boost in variance explained), across a majority of the cognitive abilities tested. Further analysis found that diffusion and brain surface properties contribute the most to the predictive power. Our findings establish a lower bound to predict individual differences in cognition using multiple neuroimaging measures of brain architecture, both structural and functional, quantify the relative predictive power of the different imaging modalities, and reveal how each modality provides unique and complementary information about individual differences in cognitive function.Author summaryCognition is a complex and interconnected process whose underlying mechanisms are still unclear. In order to unravel this question, studies usually look at one neuroimaging modality (e.g. functional MRI) and associate the observed brain properties with individual differences in cognitive performance. However, this approach is limiting because it fails to incorporate other sources of brain information and does not generalize well to new data. Here we tackled both problems by using out-of-sample testing and a multi-level learning approach that can efficiently integrate across simultaneous brain measurements. We tested this scenario by evaluating individual differences across several cognitive domains, using five measures that represent morphological, functional and structural aspects of the brain network architecture. We predicted individual cognitive differences using each brain property group separately and then stacked these predictions, forming a new matrix with as many columns as separate brain measurements, that was then fit using a regularized regression model that isolated unique information among modalities and substantially helped enhance prediction accuracy across most of the cognitive domains. This holistic approach provides a framework for capturing non-redundant variability across different imaging modalities, opening a window to easily incorporate more sources of brain information to further understand cognitive function.

Longitudinally stable, brain-based predictive models mediate the relationships between childhood cognition and socio-demographic, psychological and genetic factors

“…Accordingly, combining across modalities, as long as they are available, in one model seems to capture additional variance in the test data that cannot be achieved even by the best modality in the model. This is consistent with previous studies showing the enhanced predictive power of the stacked mode 12,21 .…”

“…It is possible, for instance, that the G-Factor depends not only on the activity of certain areas during certain cognitive tasks, such as working memory 11,12 , (task-based functional MRI; task-fMRI) but also on the intrinsic functional connectivity between different areas [13][14][15] (resting-state fMRI; rs-fMRI) as well as the anatomy of grey 16 (structural MRI; sMRI) and white 17,18 (Diffusion Tensor Imaging; DTI) matter. Recent studies in adults 19,20 have shown advantages of the multimodal approach in the prediction of the G-Factor that might outweigh the higher complexity of the models. Still, few, if any, multimodal studies have been done in children.…”

“…It is possible, for instance, that the G-Factor depends not only on the activity of certain areas during certain cognitive tasks, such as working memory 13,14 (task-based functional MRI; task-fMRI) but also on the intrinsic functional connectivity between different areas [15][16][17] (resting-state fMRI; rs-fMRI) as well as the anatomy of grey 18 (structural MRI; sMRI) and white 19,20 (Diffusion Tensor Imaging; DTI) matter. Here we built on a recent 'stacking' framework used in research with adults 21,22 that integrates information across MRI modalities into a stacked model. We then tested if the stacked model indeed enhanced predictive performance over single modalities in predicting children's G-Factor.…”

Brain structure-function coupling provides signatures for task decoding and individual fingerprinting

“…The spatial pattern of structure-function dependencies relating to cognition presented similarities both with the decoding network in lower-order somatomotor and association cortices, intrinsically characterized by strong structure-function coupling 25 , as well as with the fingerprinting network in fronto-parietal regions, characterized by weak structure-function interplay 25 . Recent work showed that structural and functional connectivities present distinct patterns of inter-individual variance as they relate to cognition 39,40 . Intriguingly, our results extend these findings identifying in the structure-function dependency a possible link between (divergent) structural and functional connectivity patterns in predicting behavior.…”

Integrating Multiple Imaging Modalities does not Boost Prediction of Carotid Artery Intima-Media Thickness in Midlife Adults

“…We adopted a transmodal approach to stacking learning for prediction of CA-IMT 18,19 . In machine learning, stacking is classified as an ensemble learning method and involves combining predictions from a set of models into a new meta feature matrix for subsequent input into a new model for final prediction 20,26 .…”