Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility

Abstract

Objectives To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software. Methods T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient. Results Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89–0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40–0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00–0.41). Conclusions Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models. Key Points • Features derived from T2W-MRI and in particular ADC differ significantly between centers when performing multicenter data analysis. • Variations in ADC are mainly (> 60%) caused by hardware and image acquisition differences and less so (< 1%) by patient- or tumor-intrinsic variations. • Features derived using different image segmentations (expert/non-expert) were reproducible, provided that whole-volume segmentations were used. When using different feature extraction software packages with similar settings, higher-order features were less reproducible.

Are We There Yet? The Value of Deep Learning in a Multicenter Setting for Response Prediction of Locally Advanced Rectal Cancer to Neoadjuvant Chemoradiotherapy

“…These observations are consistent with the findings of Schurink et al [ 45 ], who found significant intercenter variation in multicenter rectal MRI data, with greater variation in DW images and ADC maps compared to T2w images, mainly related to hardware and image acquisition protocols. They further state that such variations will likely negatively influence subsequent analysis if not corrected for [ 45 ].…”

“…These observations are consistent with the findings of Schurink et al [ 45 ], who found significant intercenter variation in multicenter rectal MRI data, with greater variation in DW images and ADC maps compared to T2w images, mainly related to hardware and image acquisition protocols. They further state that such variations will likely negatively influence subsequent analysis if not corrected for [ 45 ]. Even though imaging processing and data harmonization were applied in this study prior to network training and testing, our results indicate that further development of a dedicated data pre-processing pipeline is necessary that specifically addresses the inhomogeneities of a multicenter dataset [ 46 , 47 ].…”

Radiomic features derived from pretherapeutic MRI predict chemoradiation response in locally advanced rectal cancer

“…A study by Zhang et al 34 focusing on intensity histogram features and texture features of primary liver cancer demonstrated that most features were significantly influenced by different b-values of DWI. Schurink et al 35 found greater variations in features derived from ADC compared to T2 weighted images from 649 rectal cancer patients across nine centers. Most variation in ADC values could be explained by acquisition and scanner settings instead of essential biological differences.…”

“…Therefore, we can neither exclude features based on their interobserver reproducibility nor quantitatively evaluate the agreement between readers. However, Schurink et al 35 showed a relatively minor impact on feature reproducibility from segmentation variation and different annotation software. Lastly, there is a lack of external validation in our study.…”

Methodological considerations on segmenting rhabdomyosarcoma with diffusion-weighted imaging—What can we do better?

“…A further 20 were excluded from the review altogether for not reporting any details. This large degree of variation highlights a need for a standard for reporting details on tumor segmentation, in analogy to other established radiological practices [ 19 ].…”