Identical, but not the same: Intra-site and inter-site reproducibility of fractional anisotropy measures on two 3.0T scanners

Abstract

Diffusion Tensor Imaging (DTI) is being increasingly used to assess white matter integrity and it is therefore paramount to address the test–retest reliability of DTI measures. In this study we assessed inter- and intra-site reproducibility of two nominally identical 3 T scanners at different sites in nine healthy controls using a DTI protocol representative of typical current “best practice” including cardiac gating, a multichannel head coil, parallel imaging and optimized diffusion gradient parameters. We calculated coefficients of variation (CV) and intraclass correlation coefficients (ICC) of fractional anisotropy (FA) measures for the whole brain, for three regions of interest (ROI) and for three tracts derived from these ROI by probabilistic tracking. We assessed the impact of affine, nonlinear and template based methods for spatially aligning FA maps on the reproducibility. The intra-site CV for FA ranged from 0.8% to 3.0% with ICC from 0.90 to 0.99, while the inter-site CV ranged from 1.0% to 4.1% with ICC of 0.82 to 0.99. Nonlinear image coregistration improved reproducibility compared to affine coregistration. Normalization to template space reduced the between-subject variation, resulting in lower ICC values and indicating a possibly reduced sensitivity. CV from probabilistic tractography were about 50% higher than for the corresponding seed ROI.Reproducibility maps of the whole scan volume showed a low variation of less than 5% in the major white matter tracts but higher variations of 10–15% in gray matter regions.One of the two scanners showed better intra-site reproducibility, while the intra-site CV for both scanners was significantly better than inter-site CV. However, when using nonlinear coregistration of FA maps, the average inter-site CV was below 2%. There was a consistent inter-site bias, FA values on site 2 were 1.0–1.5% lower than on site 1. Correction for this bias with a global scaling factor reduced the inter-site CV to the range of intra-site CV. Our results are encouraging for multi-centre DTI studies in larger populations, but also illustrate the importance of the image processing pipeline for reproducibility.

Reproducibility of quantitative structural and physiological MRI measurements

“…The least consistency was observed in fornix (FX), corticospinal (CST), and superior fronto‐occipital (SFO) tracts. The lack of consistency on these three tracts can be explained by partial volume averaging and/or spatial misregistration and is similar to previous reports (Vollmar et al., 2010). The FX and CST are long, tubular white matter that passes through the areas with magnetic susceptibility and therefore prone to geometrical distortions.…”

Assessment of diffusion tensor image quality across sites and vendors using the American College of Radiology head phantom

“…With many centers contributing relatively few patients, variability between sites must be minimized. There have been several reports comparing diffusion measurements in multicenter studies 5 , 6 , 7 , 8 documenting significant variability for scanners of the same model using essentially identical technique and parameters. Many aspects of scanner performance have been reported to affect the measurement results and introduce discrepancies in the measurement (14) .…”

“…Repeatability and reproducibility data were not collected from sites participating in this study. Therefore, it is difficult to assess whether the variability of all the metrics is due to the systematic difference of machines from different vendors, as mentioned in the studies by Volimar et al, Zhu et al, Fox et al, and Teipel et al, 5 , 6 , 7 , 8 or due to random errors in the MRI measurements. Since the goal of this study was to evaluate a QC methodology for multicenter trial, knowing under what circumstance the site needs to undergo hardware/software adjustment is important.…”

“…Gradient performance and compensation of eddy currents also affect the image quality. While differences between vendors are expected, scanners of the same model and software release may also yield somewhat different results (5) as evidenced by better scanner performance after routine preventative maintenance service. These technique‐related effects may both mask and mimic biological changes in white matter.…”

“…These technique‐related effects may both mask and mimic biological changes in white matter. There have been several reports comparing diffusion measurements in multicenter studies 5 , 6 , 7 , 8 documenting significant variability for scanners of the same model using essentially identical technique and parameters. Conducting a multicenter study using quantitative analysis of diffusion tensor data requires a uniform quality control procedure to ensure that: (i) data from different sites are not biased and can be combined for statistical analysis; and (ii) data of substandard quality are excluded.…”