Repeatability of radiomics and machine learning for DWI: Short‐term repeatability study of 112 patients with prostate cancer

Abstract

Purpose To evaluate repeatability of prostate DWI‐derived radiomics and machine learning methods for prostate cancer (PCa) characterization. Methods A total of 112 patients with diagnosed PCa underwent 2 prostate MRI examinations (Scan1 and Scan2) performed on the same day. DWI was performed using 12 b‐values (0–2000 s/mm2), post‐processed using kurtosis function, and PCa areas were annotated using whole mount prostatectomy sections. A total of 1694 radiomic features including Sobel, Kirch, Gradient, Zernike Moments, Gabor, Haralick, CoLIAGe, Haar wavelet coefficients, 3D analogue to Laws features, 2D contours, and corner detectors were calculated. Radiomics and 4 feature pruning methods (area under the receiver operator characteristic curve, maximum relevance minimum redundancy, Spearman’s ρ, Wilcoxon rank‐sum) were evaluated in terms of Scan1‐Scan2 repeatability using intraclass correlation coefficient (ICC)(3,1). Classification performance for clinically significant and insignificant PCa with Gleason grade groups 1 versus >1 was evaluated by area under the receiver operator characteristic curve in unseen random 30% data split. Results The ICC(3,1) values for conventional radiomics and feature pruning methods were in the range of 0.28–0.90. The machine learning classifications varied between Scan1 and Scan2 with % of same class labels between Scan1 and Scan2 in the range of 61–81%. Surface‐to‐volume ratio and corner detector‐based features were among the most represented features with high repeatability, ICC(3,1) >0.75, consistently high ranking using all 4 feature pruning methods, and classification performance with area under the receiver operator characteristic curve >0.70. Conclusion Surface‐to‐volume ratio and corner detectors for prostate DWI led to good classification of unseen data and performed similarly in Scan1 and Scan2 in contrast to multiple conventional radiomic features.

Image Contrast, Image Pre-Processing, and T1 Mapping Affect MRI Radiomic Feature Repeatability in Patients with Colorectal Cancer Liver Metastases

“…Across all datasets, Voxel Volume and Mesh Volume were the most repeatable features. As a class, Shape features tended to have the highest ICCs, consistent with observations on T W images of cervical tumours [ 15 ], quantitative diffusion kurtosis maps of prostate tumours [ 20 ], and quantitative apparent diffusion coefficient maps of liver metastases and ovarian tumours [ 19 ]. For T W images in rectal cancer, Gourtsoyianni et al note that Gray Level Size Zone Matrix and Neighbouring Gray Tone Difference Matrix features tended to have poor repeatability [ 12 ].…”

“…By assessing MR radiomic feature repeatability using two different metrics in a relatively large clinical cohort, investigating the effects of MR sequence, image normalisation, and assumptions about feature distributions, this work contributes to the technical validation of radiomic features. By focussing on liver metastases, and using quantitative T maps and post-contrast T W images, this work complements existing repeatability studies using other MR sequences in other tumour types [ 12 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ].…”

“…By having up to a week between repeat scans for some patients, it is possible that tumours may have undergone genuine macroscopic or microscopic changes, which radiomic features may be sensitive to. This could lead to an underestimate of feature repeatability, and could be mitigated by performing test-retest studies with features evaluated from scans performed minutes apart [ 15 , 16 , 20 ]; note however that this would not be feasible for T W post-contrast scans, as the contrast agent administration cannot be repeated in such a short time window. As liver motion could affect the comparison between different sequences, future work could investigate the impact of applying motion correction before extracting radiomic features from multiple sequences.…”

“…In terms of technical validation, measuring repeatability in single centres and reproducibility across multiple centres is crucial, and provides an important step in developing metrology standards for quantitative imaging biomarkers in general [ 9 ], including radiomic features [ 10 ]. While a number of studies have assessed the repeatability and/or reproducibility of CT-derived cancer radiomic features [ 11 ], there are fewer studies investigating the repeatability of MR-derived radiomic features [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. These studies are limited by small patient numbers (≤17, with the exception of Kickingereder et al, [ 13 ] and Merisaari et al [ 20 ], with 55 and 112 patients, respectively), and the use of only one or two MR sequences (except in [ 13 ] where three were used).…”

“…While a number of studies have assessed the repeatability and/or reproducibility of CT-derived cancer radiomic features [ 11 ], there are fewer studies investigating the repeatability of MR-derived radiomic features [ 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 , 21 ]. These studies are limited by small patient numbers (≤17, with the exception of Kickingereder et al, [ 13 ] and Merisaari et al [ 20 ], with 55 and 112 patients, respectively), and the use of only one or two MR sequences (except in [ 13 ] where three were used). Furthermore, given the difficulty in directly comparing MR signal intensities from different scans, the effect of image normalisation on repeatability needs to be considered for different MR sequences, as does the validity of assumptions underlying the statistical analysis of repeatability.…”