Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers

Deist, Timo M.; Dankers, Frank J. W. M.; Valdés, Gilmer; Wijsman, Robin; Hsu, I‐Chow; Oberije, Cary; Lustberg, Tim; Soest, Johan van; Hoebers, Frank; Jochems, Arthur; Naqa, Issam El; Wee, Leonard; Morin, Olivier; Raleigh, David R.; Bots, Wouter; Kaanders, Johannes H.A.M.; Belderbos, J.; Kwint, Margriet; Solberg, Timothy D.; Monshouwer, R.; Bussink, Johan; Dekker, André; Lambin, Philippe

doi:10.1002/mp.12967

Cited by 251 publications

(195 citation statements)

References 25 publications

Supporting

Mentioning

177

Contrasting

Unclassified

Order By: Relevance

“…A number of machine learning (ML) algorithms can provide robust means to identify a subset of features to combine into a multiparametric model (24). Although several ML algorithms, alone or in combination, have been used in radiomics analysis for feature selection and classification, there is no "one fits all" approach as performance of various ML workflows have been shown to depend on application and/or type of data (25)(26)(27) Previous studies have tested cross-combination of different ML approaches, and have suggested distinct ML algorithms that depict high performance for feature selection and classification (24,25,28).…”

Section: Introductionmentioning

confidence: 99%

Next-Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Algorithms

Shiri¹,

Maleki²,

Hajianfar³

et al. 2020

Mol Imaging Biol

101

View full text Add to dashboard Cite

Conflicts of Interest:The authors declare no potential conflicts of interest. Abstract PurposeConsiderable progress has been made in assessment and management of NSCLC patients based on mutation status in the EGFR and KRAS. At the same time, NSCLC management through KRAS and EGFR mutation profiling faces some challenges. In the present work, we aimed to evaluate a comprehensive radiomics framework that enabled prediction of EGFR and KRAS mutation status in NSCLC cancer patients based on low dose CT, diagnostic CT, PET modalities radiomic features and machine learning (ML) algorithms. MethodsOur study involved NSCLC cancer patient with 186 PET, and 175 low dose CT and CTD images. More than twenty thousand radiomic features from different image-feature sets were extracted. Conventional clinically used standard uptake value (SUV) parameters were also obtained for PET images. Feature value was normalized to obtain Z-scores, followed by student t-test students for comparison, high correlated features were eliminated and the False discovery rate (FDR) correction were performed and q-value were reported for univariate analysis. Six feature selection methods and twelve classifiers were used to predict gene status in patient. We performed 10-fold cross-validation for model tuning to improve robustness and all model evaluation was reported on independent validation sets (68 patients). The mean area under the receiver operator characteristic (ROC) curve (AUC) was obtained for performance evaluation. ResultsThe best predictive power of conventional PET parameters was achieved by SUVpeak (AUC: 0.69, P-value = 0.0002) and MTV (AUC: 0.55, P-value = 0.0011) for EGFR and KRAS, respectively. Univariate analysis of extracted radiomics features improved prediction power up to AUC: 75 (q-value: 0.003, Short Run Emphasis feature of GLRLM from LOG preprocessed image of PET with sigma value 1.5) and AUC: 0.71 (q-value 0.00005, The Large Dependence Low Gray Level Emphasis from GLDM in LOG preprocessed image of CTD sigma value 5) for EGFR and KRAS, respectively. Furthermore, the machine learning algorithm improved the perdition power up to AUC: 0.82 for EGFR (LOG preprocessed of PET image set with sigma 3 with VT feature selector and SGD classifier) and AUC: 0.83 for KRAS (CT image set with sigma 3.5 with SM feature selector and SGD classifier). ConclusionOur findings demonstrated that non-invasive and reliable radiomics analysis can be successfully used to predict EGFR and KRAS mutation status in NSCLC patients. We demonstrated that radiomic features extracted from different image-feature sets could be used for EGFR and KRAS mutation status prediction in NSCLC patients, and showed that they have more predictive power than conventional imaging parameters.This study was conducted on 211 NSCLC cancer patients with available imaging and genomic data. Imaging modalities, including diagnostic CT (CTD) and PET/CT (i.e., low-dose CT (CT) used for PET attenuation correction and the PET image) for all patients were obtained (29-32) . All images we...

show abstract

Section: Introductionmentioning

confidence: 99%

Next-Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Algorithms

Shiri¹,

Maleki²,

Hajianfar³

et al. 2020

Mol Imaging Biol

101

View full text Add to dashboard Cite

show abstract

“…When looking at the AUC metric alone our results were comparable to values published in other SML based EBRT prediction studies. 13,17,18,27,28 Direct comparison of models is limited, however, because most other studies used toxicity as an endpoint rather than dose tolerance. The exception being Caine et al who also investigated protocol compliance but did not provide AUC or other performance metrics.…”

Section: Discussionmentioning

confidence: 99%

“…13 In a comparison of SML methods across a large number of datasets from different departments the work by Deist and colleagues further highlights the limitations of comparing models from different datasets as it can lead to variations in AUC performance and model superiority. 27 Beyond the standard model performance metrics, altering the probability threshold for predictive classification of outcomes was also investigated. This was in response to the knowledge that many clinical intervention decisions are not made on a 50% risk probability threshold.…”

Section: Discussionmentioning

confidence: 99%

Comparison of statistical machine learning models for rectal protocol compliance in prostate external beam radiation therapy

et al. 2020

View full text Add to dashboard Cite

Purpose: Limiting the dose to the rectum can be one of the most challenging aspects of creating a dosimetric external beam radiation therapy (EBRT) plan for prostate cancer treatment. Rectal sparing devices such as hydrogel spacers offer the prospect of increased space between the prostate and rectum, causing reduced rectal dose and potentially reduced injury. This study sought to help identify patients at higher risk of developing rectal injury based on estimated rectal dosimetry compliance prior to the EBRT simulation and planning procedure. Three statistical machine learning methods were compared for their ability to predict rectal dose outcomes with varied classification thresholds applied. Methods: Prostate cancer patients treated with conventionally fractionated EBRT to a reference dose of 74-78 Gy were invited to participate in the study. The dose volume histogram data from each dosimetric plan was used to quantify planned rectal volume receiving 50%, 83% 96%, and 102% of the reference dose. Patients were classified into two groups for each of these dose levels: either meeting tolerance by having a rectal volume less than a clinically acceptable threshold for the dose level (Y) or violating the tolerance by having a rectal volume greater than the threshold for the dose level (N). Logistic regression, classification and regression tree, and random forest models were compared for their ability to discriminate between class outcomes. Performance metrics included area under the receiver operator characteristic curve (AUC), sensitivity, specificity, positive predictive value and negative predictive value. Finally, three classification threshold levels were evaluated for their impact on model performance. Results: A total of 176 eligible participants were recruited. Variable importance differed between model methods. Area under the receiver operator characteristic curve performance varied greatly across the different rectal dose levels and between models. Logistic regression performed best at the 83% reference dose level with an AUC value of 0.844, while random forest demonstrated best discrimination at the 96% reference dose level with an AUC value of 0.733. In addition to the standard classification probability threshold of 50%, the clinically representative threshold of 10%, and the best threshold from each AUC plot was applied to compare metrics. This showed that using a 50% threshold and the best threshold from the AUC plots yields similar results. Conversely, applying the more conservative clinical threshold of 10% maximized the sensitivity at V83_RD and V96_RD for all model types. Based on the combination of the metrics, logistic regression would be the recommendation for rectal protocol compliance prediction at the 83% reference dose level, and random forest for the 96% reference dose level, particularly when using the clinical probability threshold of 10%. Conclusions: This study demonstrated the efficacy of statistical machine learning models on rectal protocol compliance prediction for prostate cancer...

show abstract

“…For example, individual-level treatment response prediction has been studied for schizophrenia [5] and depression [33]. An empirical comparison of classifiers for treatment-response prediction for chemoradiotherapy appears in [9]. Topics studied in social sciences include the effect of a discrete treatment, years of education, on an individuals' income [6], and allowing a response to depend on social interactions and treatments for other individuals [24].…”

Section: Related Workmentioning

confidence: 99%

Errors-in-Variables Modeling of Personalized Treatment-Response Trajectories

Zhang

Ashrafi

Juuti

et al. 2021

IEEE J. Biomed. Health Inform.

View full text Add to dashboard Cite

Estimating the effect of a treatment on a given outcome, conditioned on a vector of covariates, is central in many applications. However, learning the impact of a treatment on a continuous temporal response, when the covariates suffer extensively from measurement error and even the timing of the treatments is uncertain, has not been addressed. We introduce a novel data-driven method that can estimate treatment-response trajectories in this challenging scenario. We model personalized treatment-response curves as a combination of parametric response functions, hierarchically sharing information across individuals, and a sparse Gaussian process for the baseline trend. Importantly, our model considers measurement error not only in treatment covariates, but also in treatment times, a problem which arises in practice for example when treatment information is based on self-reporting. In a challenging and timely problem of estimating the impact of diet on continuous blood glucose measurements, our model leads to significant improvements in estimation accuracy and prediction.

show abstract

Machine learning algorithms for outcome prediction in (chemo)radiotherapy: An empirical comparison of classifiers

Cited by 251 publications

References 25 publications

Next-Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Algorithms

Next-Generation Radiogenomics Sequencing for Prediction of EGFR and KRAS Mutation Status in NSCLC Patients Using Multimodal Imaging and Machine Learning Algorithms

Comparison of statistical machine learning models for rectal protocol compliance in prostate external beam radiation therapy

Errors-in-Variables Modeling of Personalized Treatment-Response Trajectories

Contact Info

Product

Resources

About