The stability of imaging biomarkers in radiomics: a framework for evaluation

Abstract

This paper studies the sensitivity of a range of image texture parameters used in radiomics to: i) the number of intensity levels, ii) the method of quantisation to select the intensity levels and iii) the use of an intensity threshold. 43 commonly used texture features were studied for the gross target volume outlined on the CT component of PET/CT scans of 50 patients with non-small cell lung carcinoma (NSCLC). All cases were quantised for all values between 4 and 128 intensity levels using four commonly used quantisation methods. All results were analysed with and without a threshold range of-200 HU to 300 HU. Cases were ranked for each texture feature and for all quantisation methods with the Spearman's rank correlation coefficient determined to evaluate stability. Results showed large fluctuations in ranking, particularly for low numbers of levels, differences between quantisation methods and with the use of a threshold, with values Spearman's Rank Correlation for many parameters below 0.2. Our results demonstrated the sensitivity of radiomics features to the parameters used during analysis and highlight the risk of low reproducibility comparing studies with slightly different parameters. In terms of the lung cancer CT datasets, this study supports the use of 128 intensity levels, the same uniform quantiser applied to all scans and thresholding of the data. It also supports several of the features recommended in the literature for such studies such as skewness and kurtosis. A recommended framework is presented for curation of the data analysis process to ensure stability of results.

T2w‐MRI signal normalization affects radiomics features reproducibility

“…This is in line with recent publications that highlighted some vulnerabilities in the radiomic signature development, related to the risk of including features that are mainly correlated to the volume in prediction models. 35,38 Reproducibility of radiomic features was here assessed using the ICC metric, able to combine information about the degree of correlation and agreement between measurements. 34 This coefficient is one of the most adopted for the estimation of repeatability and reproducibility of radiomic indices, as reported in Traverso et al 8 In fact, it has been used also in the latest work considering T2w-MRI.…”

“…For this reason, it was not surprising that they are found as highly reliable, especially when volume is not modified in the considered conditions (as in the present study) or when the inter‐observer agreement is high (Dice > 0.9). This is in line with recent publications that highlighted some vulnerabilities in the radiomic signature development, related to the risk of including features that are mainly correlated to the volume in prediction models …”

Radiomics as a personalized medicine tool in lung cancer: Separating the hope from the hype

“…Results from one disease site are not necessarily transferrable to another [ 108 ]. Expert ROI definition [ 103 ], multiple observers [ 103 , 104 , 108 ], identification of stable features with respect to delineation [ 90 , 104 , 105 ], automated segmentation [ 106 , 107 ], image filtering [ 108 ] Pre-processing F Number of grey levels used to discretize histogram and texture features affects feature values [ 96 , 98 , 109 ], as does bin width [ 94 ]. Texture features can be normalized to reduce dependency on the number of grey levels [ 98 ], number of grey levels used for discretization should be recorded with feature formula.…”

“… Texture features can be normalized to reduce dependency on the number of grey levels [ 98 ], number of grey levels used for discretization should be recorded with feature formula. 128 grey levels may be optimal for texture features, along with thresholding [ 109 ] Feature extraction No studies found in the literature search. Feature correlation G Strong correlations between tumor volume and radiomic features exist [ 98 , [110] , [111] , [112] ] Normalization of features to volume [ 98 ], bit depth resampling [ 110 ], feature redesign [ 110 ], more robust statistics to check added value of radiomics signatures [ 111 ].…”

Narrative review of radiomics for classifying pulmonary nodules and potential impact on lung cancer screening

“…This is particularly true for second and higher order texture features, transform based features, and features derived from unsupervised machine learning algorithms. How intensity values of CT and PET data is grouped together (aka binning) has been shown to have significant effects in the variability of texture features with optimal number of intensity values of 128 showing the least variability (38,39). Radiomic texture features derived from GLCM may be more robust compared to features derived from other matrices (40).…”