A Comprehensive Machine Learning Benchmark Study for Radiomics-Based Survival Analysis of CT Imaging Data in Patients With Hepatic Metastases of CRC

Abstract

Objectives Optimizing a machine learning (ML) pipeline for radiomics analysis involves numerous choices in data set composition, preprocessing, and model selection. Objective identification of the optimal setup is complicated by correlated features, interdependency structures, and a multitude of available ML algorithms. Therefore, we present a radiomics-based benchmarking framework to optimize a comprehensive ML pipeline for the prediction of overall survival. This study is conducted on an image set of patients with hepatic metastases of colorectal cancer, for which radiomics features of the whole liver and of metastases from computed tomography images were calculated. A mixed model approach was used to find the optimal pipeline configuration and to identify the added prognostic value of radiomics features. Materials and Methods In this study, a large-scale ML benchmark pipeline consisting of preprocessing, feature selection, dimensionality reduction, hyperparameter optimization, and training of different models was developed for radiomics-based survival analysis. Portal-venous computed tomography imaging data from a previous prospective randomized trial evaluating radioembolization of liver metastases of colorectal cancer were quantitatively accessible through a radiomics approach. One thousand two hundred eighteen radiomics features of hepatic metastases and the whole liver were calculated, and 19 clinical parameters (age, sex, laboratory values, and treatment) were available for each patient. Three ML algorithms—a regression model with elastic net regularization (glmnet), a random survival forest (RSF), and a gradient tree-boosting technique (xgboost)—were evaluated for 5 combinations of clinical data, tumor radiomics, and whole-liver features. Hyperparameter optimization and model evaluation were optimized toward the performance metric integrated Brier score via nested cross-validation. To address dependency structures in the benchmark setup, a mixed-model approach was developed to compare ML and data configurations and to identify the best-performing model. Results Within our radiomics-based benchmark experiment, 60 ML pipeline variations were evaluated on clinical data and radiomics features from 491 patients. Descriptive analysis of the benchmark results showed a preference for RSF-based pipelines, especially for the combination of clinical data with radiomics features. This observation was supported by the quantitative analysis via a linear mixed model approach, computed to differentiate the effect of data sets and pipeline configurations on the resulting performance. This revealed the RSF pipelines to consistently perform similar or better than glmnet and xgboost. Further, for the RSF, there was no significantly better-performing pipeline composition regarding the sort of preprocessing or hyperparameter optimization. Conclusions Our study introduces a benchmark framework for radiomics-based survival analysis, aimed at identifying the optimal settings with respect to different radiomics data sources and various ML pipeline variations, including preprocessing techniques and learning algorithms. A suitable analysis tool for the benchmark results is provided via a mixed model approach, which showed for our study on patients with intrahepatic liver metastases, that radiomics features captured the patients' clinical situation in a manner comparable to the provided information solely from clinical parameters. However, we did not observe a relevant additional prognostic value obtained by these radiomics features.

Addressing the Contrast Media Recognition Challenge

“…In addition, the study's findings suggest that the models could be used for automatically selecting the appropriate CT for running specific AI algorithms, also known as AI orchestration. This automatic selection of identifying CT phases could be used, for example, to decide which segmentation algorithms are appropriate, 12–15 for tumor characterization of hepatic metastases, 16 for the characterization of healthy or tumor tissue in the kidneys, 17 and for predicting therapy response in gastric cancer 18 …”

“…This aspect is particularly important for the orchestration of artificial intelligence (AI) applications, ensuring that a CT scan is accurately channeled to compatible AI systems. Delivering the correct contrast phase is important for many different types of AI systems including segmentation models that rely on a specific contrast phase, 12–15 for the automated characterization of tissues 16,17 and to predict therapeutic response 18 …”

Machine Learning Combined with Radiomics Facilitating the Personal Treatment of Malignant Liver Tumors

“…Stüber et al [14] collected CT images from 491 CLM patients who underwent TARE to extract radiomics features and create models. Nevertheless, they did not observe significant additional prognostic value in these radiomics features for predicting overall survival when compared to information obtained solely from clinical parameters.…”

Advancing NSCLC pathological subtype prediction with interpretable machine learning: a comprehensive radiomics-based approach

“…The term "radiomics" refers to the automated or semi-automatic postprocessing techniques used to analyze various features extracted from imaging exams, and reveals the correlation between these quantitative features and clinical histology or biomarkers (11). In recent years, numerous research have validated the great potential of machine learning combined with radiomics for accurate recognition of histological subtypes, molecular subtypes, and clinical outcome prediction (12)(13)(14). Similarly, radiomics has demonstrated comparable success in identifying lung cancer pathological subtypes in previous studies (15,16).…”