Joint Graph Convolution for Analyzing Brain Structural and Functional Connectome

Abstract

The white-matter (micro-)structural architecture of the brain promotes synchrony among neuronal populations, giving rise to richly patterned functional connections. A fundamental problem for systems neuroscience is determining the best way to relate structural and functional networks quantified by diffusion tensor imaging and resting-state functional MRI. As one of the state-of-the-art approaches for network analysis, graph convolutional networks (GCN) have been separately used to analyze functional and structural networks, but have not been applied to explore inter-network relationships. In this work, we propose to couple the two networks of an individual by adding inter-network edges between corresponding brain regions, so that the joint structure-function graph can be directly analyzed by a single GCN. The weights of inter-network edges are learnable, reflecting non-uniform structure-function coupling strength across the brain. We apply our Joint-GCN to predict age and sex of 662 participants from the public dataset of the National Consortium on Alcohol and Neurodevelopment in Adolescence (NCANDA) based on their functional and micro-structural white-matter networks. Our results support that the proposed Joint-GCN outperforms existing multi-modal graph learning approaches for analyzing structural and functional networks.

Predicting gender from structural and functional connectomes via brain and population graph neural networks

“…We compared BrainGAT against logistic regression, dGLCN [9], and Joint-GCN [4]. For logistic regression, we applied both L1 and L2 regularization, with a ratio of 0.5 and a regularization strength of 1.0.…”

“…Unlike existing work, BrainGAT enables both SC and FC matrices to be used simultaneously, allowing cross-modal interactions to be considered. Our method outperformed baselines approaches as well as existing multimodal modelling [4] and dual graph [9] approaches. Ablation studies reaffirm the value of combining brain graphs and population graphs in both single modality and multimodal settings.…”

“…Existing multimodal neuroimaging analysis [4], [5] typically combines diffusion tensor imaging (DTI) and fMRI, which can be represented as structural connectivity (SC) and functional connectivity (FC) matrices. Recent works on training deep learning models on such connectome datasets have converged towards the use of graph neural networks (GNN).…”

Encoding and decoding multimodal brain images for disease biomarker discovery

“…In early/data-level fusion, raw features or features learnt from processing unimodal data (e.g. volumetric measures from MRI, di↵usion measures from DTI and correlation measures from fMRI) are joined together into a single feature vector [67] or graph [68]. In late fusion, various forms of voting mechanisms (e.g.…”

“…Existing multimodal neuroimaging analysis [68,184] typically combines DTI and fMRI, which can be represented as SC and FC matrices. Recent works on training deep learning models on such connectome datasets have converged towards the use of GNN.…”