Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets

McNitt‐Gray, Michael F.; Napel, Sandy; Jaggi, Akshay; Mattonen, Sarah A.; Hadjiiski, Lubomir M.; Muzi, Mark; Goldgof, Dmitry B.; Balagurunathan, Yoganand; Pierce, Larry A.; Kinahan, Paul E.; Jones, Ella F.; Nguyen, Anwell; Virkud, Apurva; Chan, Heang Ping; Emaminejad, Nastaran; Wahi-Anwar, Muhammad; Daly, Megan E.; Abdalah, Mahmoud A.; Yang, Hao; Lin, Liwu; Lv, Wenzhi; Rahmim, Arman; Gastounioti, Aimilia; Pati, Sarthak; Bakas, Spyridon; Kontos, Despina; Zhao, Binsheng; Kalpathy–Cramer, Jayashree; Farahani, Keyvan

doi:10.18383/j.tom.2019.00031

Cited by 66 publications

(54 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research has investigated sources of variation in feature computation (81,(102)(103)(104). Two collaborative studies on this topic are reviewed and discussed in the following subsections.…”

Section: Studies Exploring and Reducing Variability In Feature Computmentioning

confidence: 99%

“…This study, conducted by ten teams from the PET/CT working group of the QIN funded by the National Cancer Institute (NCI), explored the agreement of 13 software packages on nine basic radiomics features including volume, 2D and 3D diameters, mean density, standard deviation, kurtosis, surface area, sphericity, and GLCM entropy ( 103 ). The investigators applied the feature extraction software used by the teams (about half open source and half in-house) to both Digital Reference Objects (DROs) and patient image data.…”

Section: Feature Extractionmentioning

confidence: 99%

See 1 more Smart Citation

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Zhao

2021

Front. Oncol.

View full text Add to dashboard Cite

Radiomics is the method of choice for investigating the association between cancer imaging phenotype, cancer genotype and clinical outcome prediction in the era of precision medicine. The fast dispersal of this new methodology has benefited from the existing advances of the core technologies involved in radiomics workflow: image acquisition, tumor segmentation, feature extraction and machine learning. However, despite the rapidly increasing body of publications, there is no real clinical use of a developed radiomics signature so far. Reasons are multifaceted. One of the major challenges is the lack of reproducibility and generalizability of the reported radiomics signatures (features and models). Sources of variation exist in each step of the workflow; some are controllable or can be controlled to certain degrees, while others are uncontrollable or even unknown. Insufficient transparency in reporting radiomics studies further prevents translation of the developed radiomics signatures from the bench to the bedside. This review article first addresses sources of variation, which is illustrated using demonstrative examples. Then, it reviews a number of published studies and progresses made to date in the investigation and improvement of feature reproducibility and model performance. Lastly, it discusses potential strategies and practical considerations to reduce feature variability and improve the quality of radiomics study. This review focuses on CT image acquisition, tumor segmentation, quantitative feature extraction, and the disease of lung cancer.

show abstract

“…Research has investigated sources of variation in feature computation (81,(102)(103)(104). Two collaborative studies on this topic are reviewed and discussed in the following subsections.…”

Section: Studies Exploring and Reducing Variability In Feature Computmentioning

confidence: 99%

Section: Feature Extractionmentioning

confidence: 99%

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Zhao

2021

Front. Oncol.

View full text Add to dashboard Cite

show abstract

“…If one centre determines that having a certain feature value above a given threshold is predictive of malignancy in lung nodule screening, a second one can reuse that result only if (a) the features are computed using the same settings or (b) the features are stable enough. Concerning intensity quantisation, of course one sensible approach would be to stick to one value (

is a common choice [ 16 , 62 , 63 , 64 ]) in order to have comparable data. However, for some features simple mathematical transformations could be applied to make the features independent of the number of quantisation levels (see for instance Appendix A ).…”

Section: Discussionmentioning

confidence: 99%

Impact of Lesion Delineation and Intensity Quantisation on the Stability of Texture Features from Lung Nodules on CT: A Reproducible Study

et al. 2021

View full text Add to dashboard Cite

Computer-assisted analysis of three-dimensional imaging data (radiomics) has received a lot of research attention as a possible means to improve the management of patients with lung cancer. Building robust predictive models for clinical decision making requires the imaging features to be stable enough to changes in the acquisition and extraction settings. Experimenting on 517 lung lesions from a cohort of 207 patients, we assessed the stability of 88 texture features from the following classes: first-order (13 features), Grey-level Co-Occurrence Matrix (24), Grey-level Difference Matrix (14), Grey-level Run-length Matrix (16), Grey-level Size Zone Matrix (16) and Neighbouring Grey-tone Difference Matrix (five). The analysis was based on a public dataset of lung nodules and open-access routines for feature extraction, which makes the study fully reproducible. Our results identified 30 features that had good or excellent stability relative to lesion delineation, 28 to intensity quantisation and 18 to both. We conclude that selecting the right set of imaging features is critical for building clinical predictive models, particularly when changes in lesion delineation and/or intensity quantisation are involved.

show abstract

“…SERA calculates up to 487 standardized radiomic features aiming to standardize the preprocessing and feature evaluation phases, and to meet standards by the IBSI, in order to contribute to reproducible research [39]. Some of the radiomic features by SERA have also been validated in another standardization study [40]. Section S.2 in Supplementary Materials includes a brief introduction to SERA and its configuration, in addition to a spreadsheet of computed features from IBSI benchmark phantoms.…”

Section: Radiomics Frameworkmentioning

confidence: 99%

Radiomics Analysis of Clinical Myocardial Perfusion Stress SPECT Images to Identify Coronary Artery Calcification

Ashrafinia

Dalaie

et al. 2021

Preprint

View full text Add to dashboard Cite

PurposeMyocardial perfusion stress SPECT (MPSS) is an established diagnostic test for patients suspected with coronary artery disease (CAD). Meanwhile, coronary artery calcification (CAC) scoring obtained from diagnostic CT is a highly specific test, offering incremental diagnostic information in identifying patients with significant CAD yet normal MPSS scans. However, after decades of wide utilization of MPSS, CAC is not commonly reimbursed (e.g. by the CMS), nor widely deployed in community settings. We aimed to perform radiomics analysis of normal MPSS scans to investigate the potential to predict the CAC score.MethodsWe collected data from 428 patients with normal (non-ischemic) MPSS (99mTc-Sestamibi; consensus reading). A nuclear medicine physician verified iteratively reconstructed images (attenuation-corrected) to be free from fixed perfusion defects and artifactual attenuation. 3D images were automatically segmented into 4 regions of interest (ROIs), including myocardium and 3 vascular segments (LAD-LCX-RCA). We used our software package, standardized environment for radiomics analysis (SERA), to extract 487 radiomic features in compliance with the image biomarker standardization initiative (IBSI). Isotropic cubic voxels were discretized using fixed bin-number discretization (8 schemes). We first performed blind-to-outcome feature selection focusing on a priori usefulness, dynamic range, and redundancy of features. Subsequently, we performed univariate and multivariate machine learning analyses to predict CAC scores from i) selected radiomic features, ii) 10 clinical features, iii) combined radiomics + clinical features. Univariate analysis invoked Spearman correlation with Benjamini-Hotchberg false-discovery correction. The multivariate analysis incorporated stepwise linear regression, where we randomly selected a 15% test set and divided the other 85% of data into 70% training and 30% validation sets. Training started from a constant (intercept) model, iteratively adding/removing features (stepwise regression), invoking Akaike information criterion (AIC) to discourage overfitting. Validation was run similarly, except that the training output model was used as the initial model. We randomized training/validation sets 20 times, selecting the best model using log-likelihood for evaluation in the test set. Assessment in the test set was performed thoroughly by running the entire operation 50 times, subsequently employing Fisher’s method to verify the significance of independent tests.ResultsUnsupervised feature selection significantly reduced 8×487 features to 56. In univariate analysis, no feature survived FDR to directly correlate with CAC scores. Applying Fisher’s method to the multivariate regression results demonstrated combining radiomics with the clinical features to enhance the significance of the prediction model across all cardiac segments. The median absolute Pearson’s coefficient values / p-values for the three feature-pools (radiomics, clinical, combined) were: (0.15, 0.38, 0.41)/(0.1, 0.001, 0.0006) for myocardium, (0.24, 0.35, 0.41)/(0.05, 0.004, 0.0007) for LAD, (0.07, 0.24, 0.28)/(0.4, 0.06, 0.02), for LCX, and (0.06, 0.16, 0.24)/(0.4, 0.2, 0.05) for RCA, demonstrating consistently enhanced correlation and significance for combined radiomics and clinical features across all cardiac segments.ConclusionsOur standardized and statistically robust multivariate analysis demonstrated significant prediction of the CAC score for all cardiac segments when combining MPSS radiomic features with clinical features, suggesting radiomics analysis can add diagnostic or prognostic value to standard MPSS for wide clinical usage.

show abstract

Standardization in Quantitative Imaging: A Multicenter Comparison of Radiomic Features from Different Software Packages on Digital Reference Objects and Patient Data Sets

Cited by 66 publications

References 22 publications

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Impact of Lesion Delineation and Intensity Quantisation on the Stability of Texture Features from Lung Nodules on CT: A Reproducible Study

Radiomics Analysis of Clinical Myocardial Perfusion Stress SPECT Images to Identify Coronary Artery Calcification

Contact Info

Product

Resources

About