Tractography based on non-invasive diffusion imaging is central to the study of human brain connectivity. To date, the approach has not been systematically validated in ground truth studies. Based on a simulated human brain data set with ground truth tracts, we organized an open international tractography challenge, which resulted in 96 distinct submissions from 20 research groups. Here, we report the encouraging finding that most state-of-the-art algorithms produce tractograms containing 90% of the ground truth bundles (to at least some extent). However, the same tractograms contain many more invalid than valid bundles, and half of these invalid bundles occur systematically across research groups. Taken together, our results demonstrate and confirm fundamental ambiguities inherent in tract reconstruction based on orientation information alone, which need to be considered when interpreting tractography and connectivity results. Our approach provides a novel framework for estimating reliability of tractography and encourages innovation to address its current limitations.

IntroductionIn patients with multiple sclerosis (MS), conventional magnetic resonance imaging (MRI) provides only limited insights into the nature of brain damage with modest clinic-radiological correlation. In this study, we applied recent advances in MRI techniques to study brain microstructural alterations in early relapsing-remitting MS (RRMS) patients with minor deficits. Further, we investigated the potential use of advanced MRI to predict functional performances in these patients.MethodsBrain relaxometry (T1, T2, T2*) and magnetization transfer MRI were performed at 3T in 36 RRMS patients and 18 healthy controls (HC). Multicontrast analysis was used to assess for microstructural alterations in normal-appearing (NA) tissue and lesions. A generalized linear model was computed to predict clinical performance in patients using multicontrast MRI data, conventional MRI measures as well as demographic and behavioral data as covariates.ResultsQuantitative T2 and T2* relaxometry were significantly increased in temporal normal-appearing white matter (NAWM) of patients compared to HC, indicating subtle microedema (P = 0.03 and 0.004). Furthermore, significant T1 and magnetization transfer ratio (MTR) variations in lesions (mean T1 z-score: 4.42 and mean MTR z-score: −4.09) suggested substantial tissue loss. Combinations of multicontrast and conventional MRI data significantly predicted cognitive fatigue (P = 0.01, Adj-R2 = 0.4), attention (P = 0.0005, Adj-R2 = 0.6), and disability (P = 0.03, Adj-R2 = 0.4).ConclusionAdvanced MRI techniques at 3T, unraveled the nature of brain tissue damage in early MS and substantially improved clinical–radiological correlations in patients with minor deficits, as compared to conventional measures of disease.

Diffusion MRI fiber tractography is widely used to probe the structural connectivity of the brain, with a range of applications in both clinical and basic neuroscience. Despite widespread use, tractography has well-known pitfalls that limits the anatomical accuracy of this technique. Numerous modern methods have been developed to address these shortcomings through advances in acquisition, modeling, and computation. To test whether these advances improve tractography accuracy, we organized the 3-D Validation of Tractography with Experimental MRI (3D-VoTEM) challenge at the ISBI 2018 conference. We made available three unique independent tractography validation datasets - a physical phantom and two ex vivo brain specimens - resulting in 176 distinct submissions from 9 research groups. By comparing results over a wide range of fiber complexities and algorithmic strategies, this challenge provides a more comprehensive assessment of tractography’s inherent limitations than has been reported previously. The central results were consistent across all sub-challenges in that, despite advances in tractography methods, the anatomical accuracy of tractography has not dramatically improved in recent years. Taken together, our results independently confirm findings from decades of tractography validation studies, demonstrate inherent limitations in reconstructing white matter pathways using diffusion MRI data alone, and highlight the need for alternative or combinatorial strategies to accurately map the fiber pathways of the brain.

Fiber tractography based on non-invasive diffusion imaging is at the heart of connectivity studies of the human brain. To date, the approach has not been systematically validated in ground truth studies. Based on a simulated human brain dataset with ground truth white matter tracts, we organized an open international tractography challenge, which resulted in 96 distinct submissions from 20 research groups. While most state-of-the-art algorithms reconstructed 90% of ground truth bundles to at least some extent, on average they produced four times more invalid than valid bundles. About half of the invalid bundles occurred systematically in the majority of submissions. Our results demonstrate fundamental ambiguities inherent to tract reconstruction methods based on diffusion orientation information, with critical consequences for the approach of diffusion tractography in particular and human connectivity studies in general.