Predictive performance of radiomic models based on features extracted from pretrained deep networks

Abstract

Objectives In radiomics, generic texture and morphological features are often used for modeling. Recently, features extracted from pretrained deep networks have been used as an alternative. However, extracting deep features involves several decisions, and it is unclear how these affect the resulting models. Therefore, in this study, we considered the influence of such choices on the predictive performance. Methods On ten publicly available radiomic datasets, models were trained using feature sets that differed in terms of the utilized network architecture, the layer of feature extraction, the used set of slices, the use of segmentation, and the aggregation method. The influence of these choices on the predictive performance was measured using a linear mixed model. In addition, models with generic features were trained and compared in terms of predictive performance and correlation. Results No single choice consistently led to the best-performing models. In the mixed model, the choice of architecture (AUC + 0.016; p < 0.001), the level of feature extraction (AUC + 0.016; p < 0.001), and using all slices (AUC + 0.023; p < 0.001) were highly significant; using the segmentation had a lower influence (AUC + 0.011; p = 0.023), while the aggregation method was insignificant (p = 0.774). Models based on deep features were not significantly better than those based on generic features (p > 0.05 on all datasets). Deep feature sets correlated moderately with each other (r = 0.4), in contrast to generic feature sets (r = 0.89). Conclusions Different choices have a significant effect on the predictive performance of the resulting models; however, for the highest performance, these choices should be optimized during cross-validation.

Investigation of radiomics and deep convolutional neural networks approaches for glioma grading

“…Table 5 showed better results using DCNN (EffcientNetB0) in comparison to stateof-art methods found in [54,58]. The performance measurements obtained with radiomics using RNSCV showed the same trend as [54,58]. The Melanoma dataset presented a low AUC of 0.577 compared to other datasets using radiomics analysis; this is in accordance with the fact that physicians could also not predict the BRAF mutation staging [65].…”

“…This has the advantage of comparing radiomics and DCNN based approaches which was not performed previously. Table 5 showed better results using DCNN (EffcientNetB0) in comparison to stateof-art methods found in [54,58]. The performance measurements obtained with radiomics using RNSCV showed the same trend as [54,58].…”

“…The radiomics workflow using XGBOOST gave better results than the WORC framework [58] for LIPO, Desmoid, CRLM, and Melanoma, similar result for GIST, and was worse for LIVER. It was less performant than radiomics workflows in [54]. MIR that were computed based on MDI were found for LIPO, Desmoid and GIST only with a number of selections 29, 41, and 30 times respectively.…”

“…The programs of Demircioğlu [34,54] were used for the extraction of 2016 features for each patients. The standard features included shape features, firstorder, GLCM, NGLDM, GLRLM, GLZLM.…”

“…As in the case of the BraTS'20, experiments demonstrated that XGBOOST and EfficientNetB0 outperformed the other radiomics and DCNN approaches respectively. Table 5 shows the performance measurements and compare them with state-of-art results ( [54,58]) obtained on the same datasets (LIPO, LIVER, Desmoid, GIST, CRLM, and Melanoma). Figure 13 compares the roc curves between radiomics and DCNN approaches.…”