Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement

Park, Ji Eun; Kim, Dong-Hyun; Kim, Ho Sung; Park, Seo Young; Kim, Jung Youn; Cho, Se Jin; Shin, Jae Ho

doi:10.1007/s00330-019-06360-z

Cited by 211 publications

(182 citation statements)

References 87 publications

Supporting

Mentioning

157

Contrasting

Unclassified

Order By: Relevance

“…Overall, these results are in line with a previous study on the accuracy of reporting of radiomics in oncologic studies [18]. In that study, 77 articles from high-impact imaging journals were analyzed with regard to adherence to the TRIPOD checklist [19].…”

Section: Discussionsupporting

confidence: 89%

Artificial intelligence abstracts from the European Congress of Radiology: analysis of topics and compliance with the STARD for abstracts checklist

et al. 2020

View full text Add to dashboard Cite

Objectives: To analyze all artificial intelligence abstracts presented at the European Congress of Radiology (ECR) 2019 with regard to their topics and their adherence to the Standards for Reporting Diagnostic accuracy studies (STARD) checklist. Methods: A total of 184 abstracts were analyzed with regard to adherence to the STARD criteria for abstracts as well as the reported modality, body region, pathology, and use cases. Results: Major topics of artificial intelligence abstracts were classification tasks in the abdomen, chest, and brain with CT being the most commonly used modality. Out of the 10 STARD for abstract criteria analyzed in the present study, on average, 5.32 (SD = 1.38) were reported by the 184 abstracts. Specifically, the highest adherence with STARD for abstracts was found for general interpretation of results of abstracts (100.0%, 184 of 184), clear study objectives (99.5%, 183 of 184), and estimates of diagnostic accuracy (96.2%, 177 of 184). The lowest STARD adherence was found for eligibility criteria for participants (9.2%, 17 of 184), type of study series (13.6%, 25 of 184), and implications for practice (20.7%, 44 of 184). There was no significant difference in the number of reported STARD criteria between abstracts accepted for oral presentation (M = 5.35, SD = 1.31) and abstracts accepted for the electronic poster session (M = 5.39, SD = 1.45) (p = .86). Conclusions: The adherence with STARD for abstract was low, indicating that providing authors with the related checklist may increase the quality of abstracts.

show abstract

Section: Discussionsupporting

confidence: 89%

Artificial intelligence abstracts from the European Congress of Radiology: analysis of topics and compliance with the STARD for abstracts checklist

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The main limitations in quality were the following: no cost-effectiveness analysis (32 studies, 100%); lack of open-data repositories ( n = 31, 97%); no phantom calibration ( n = 31, 97%); failure to include a calibration statistic ( n = 30, 94%); lack of prospective design ( n = 28, 87%); and missing validation cohort ( n = 18, 56%). At the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) checklist [ 65 ] (31 elements), studies had an average score of 18 ± 3 points (range 14–29), i.e., 58 ± 10% of the maximum possible score. According to the quality assessment of diagnostic accuracy studies (QUADAS-2) [ 66 ], there was a high risk of a patient selection bias in 34% of papers because of selection/inclusion criteria in most cases.…”

Section: Resultsmentioning

confidence: 99%

“…In the case of positive evaluation, the entire reference list was manually examined to detect other potential candidate articles, which might have been left out by the search algorithm. The quality of the included studies was assessed by using the methods-related Radiometrics Qualitative Score, as proposed by Lambin et al [ 64 ], and the clinically-oriented TRIPOD checklist, as proposed by Park et al [ 65 ]. The presence of relevant bias in the included studies was evaluated according to QUADAS-2 [ 66 ].…”

Section: Methodsmentioning

confidence: 99%

Radiomics of Liver Metastases: A Systematic Review

Fiz

Viganò

Gennaro

et al. 2020

Cancers

View full text Add to dashboard Cite

Multidisciplinary management of patients with liver metastases (LM) requires a precision medicine approach, based on adequate profiling of tumor biology and robust biomarkers. Radiomics, defined as the high-throughput identification, analysis, and translational applications of radiological textural features, could fulfill this need. The present review aims to elucidate the contribution of radiomic analyses to the management of patients with LM. We performed a systematic review of the literature through the most relevant databases and web sources. English language original articles published before June 2020 and concerning radiomics of LM extracted from CT, MRI, or PET-CT were considered. Thirty-two papers were identified. Baseline higher entropy and lower homogeneity of LM were associated with better survival and higher chemotherapy response rates. A decrease in entropy and an increase in homogeneity after chemotherapy correlated with radiological tumor response. Entropy and homogeneity were also highly predictive of tumor regression grade. In comparison with RECIST criteria, radiomic features provided an earlier prediction of response to chemotherapy. Lastly, texture analyses could differentiate LM from other liver tumors. The commonest limitations of studies were small sample size, retrospective design, lack of validation datasets, and unavailability of univocal cut-off values of radiomic features. In conclusion, radiomics can potentially contribute to the precision medicine approach to patients with LM, but interdisciplinarity, standardization, and adequate software tools are needed to translate the anticipated potentialities into clinical practice.

show abstract

“…Compared to the first application of the RQS, the average RQS rating was lower (10.8% vs 21.9%) as was the rating for the best performing study (48% vs 55.5%). Another, recently published review employing the RQS did not report inter-rater agreement [25].Only few systematic reviews in radiomics literature have been published and even fewer assessed methodological quality systematically and quantitatively. As a result, the RQS has not yet found widespread application.…”

Section: Discussionmentioning

confidence: 99%

Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma—a systematic review and meta-analysis

et al. 2020

View full text Add to dashboard Cite

Objectives (1) To assess the methodological quality of radiomics studies investigating histological subtypes, therapy response, and survival in patients with renal cell carcinoma (RCC) and (2) to determine the risk of bias in these radiomics studies. Methods In this systematic review, literature published since 2000 on radiomics in RCC was included and assessed for methodological quality using the Radiomics Quality Score. The risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies tool and a meta-analysis of radiomics studies focusing on differentiating between angiomyolipoma without visible fat and RCC was performed. Results Fifty-seven studies investigating the use of radiomics in renal cancer were identified, including 4590 patients in total. The average Radiomics Quality Score was 3.41 (9.4% of total) with good inter-rater agreement (ICC 0.96, 95% CI 0.93-0.98). Three studies validated results with an independent dataset, one used a publically available validation dataset. None of the studies shared the code, images, or regions of interest. The meta-analysis showed moderate heterogeneity among the included studies and an odds ratio of 6.24 (95% CI 4.27-9.12; p < 0.001) for the differentiation of angiomyolipoma without visible fat from RCC. Conclusions Radiomics algorithms show promise for answering clinical questions where subjective interpretation is challenging or not established. However, the generalizability of findings to prospective cohorts needs to be demonstrated in future trials for progression towards clinical translation. Improved sharing of methods including code and images could facilitate independent validation of radiomics signatures. Key Points • Studies achieved an average Radiomics Quality Score of 10.8%. Common reasons for low Radiomics Quality Scores were unvalidated results, retrospective study design, absence of open science, and insufficient control for multiple comparisons. • A previous training phase allowed reaching almost perfect inter-rater agreement in the application of the Radiomics Quality Score. • Meta-analysis of radiomics studies distinguishing angiomyolipoma without visible fat from renal cell carcinoma show moderate diagnostic odds ratios of 6.24 and moderate methodological diversity.

show abstract

Quality of science and reporting of radiomics in oncologic studies: room for improvement according to radiomics quality score and TRIPOD statement

Cited by 211 publications

References 87 publications

Artificial intelligence abstracts from the European Congress of Radiology: analysis of topics and compliance with the STARD for abstracts checklist

Artificial intelligence abstracts from the European Congress of Radiology: analysis of topics and compliance with the STARD for abstracts checklist

Radiomics of Liver Metastases: A Systematic Review

Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma—a systematic review and meta-analysis

Contact Info

Product

Resources

About