Machine learning (ML) algorithms for selecting and combining radiomic features into multiparametric prediction models have become popular; however, it has been shown that large variations in performance can be obtained by relying on different approaches. The purpose of this study was to evaluate the potential benefit of combining different algorithms into an improved consensus for the final prediction, as it has been shown in other fields. Methods: The evaluation was carried out in the context of the use of radiomics from 18F-FDG PET/CT images for predicting outcome in stage II-III Non-Small Cell Lung Cancer. A cohort of 138 patients was exploited for the present analysis. Eighty-seven patients had been previously recruited retrospectively for another study and were used here for training and internal validation. We also used data from prospectively recruited patients (n = 51) for testing. Three different machine learning pipelines relying on embedded feature selection were trained to predict overall survival (OS) as a binary classification: Support Vector machines (SVMs), Random Forests (RFs), and Logistic Regression (LR). Two different clinical endpoints were investigated: median OS or OS shorter than 6 months. The fusion of the three approaches was implemented using two different strategies: majority voting on the binary outputs or averaging of the output probabilities. Results: Our results confirm previous findings, highlighting that different ML pipelines select different sets of features and reach different classification performances (accuracy in the testing set ranging between 63% and 67% for median OS, and between 75% and 80% for OS < 6 months). Generating a consensus improved the performance for both endpoints; with the probabilities averaging strategy outperforming the majority voting (accuracy of 78% vs. 71% for median OS and 89 vs. 84% for OS < 6 months). Overall, the performance of these radiomic-based models outperformed the standard clinical staging in both endpoints (accuracy of 58% and 53% accuracy in the testing set for each endpoint). Conclusion: Although obtained in a small cohort of patients, our results suggest that a consensus of machine learning algorithms can improve performance in the context of radiomics. The resulting prognostic stratification in the prospective testing cohort is higher than when relying on the clinical stage. This could be of interest for clinical practice as it could help to identify patients with higher risk amongst stage II and III patients, who could benefit from intensified treatment and/or more frequent follow-up after treatment.

BackgroundThe aim of this work was to investigate the ability of building prognostic models in non-small cell lung cancer (NSCLC) using radiomic features from positron emission tomography and computed tomography with 2-deoxy-2-[fluorine-18]fluoro-d-glucose (18F-FDG PET/CT) images based on a “rough” volume of interest (VOI) containing the tumor instead of its accurate delineation, which is a significant time-consuming bottleneck of radiomics analyses.MethodsA cohort of 138 patients with stage II–III NSCLC treated with radiochemotherapy recruited retrospectively (n = 87) and prospectively (n = 51) was used. Two approaches were compared: firstly, the radiomic features were extracted from the delineated primary tumor volumes in both PET (using the automated fuzzy locally adaptive Bayesian, FLAB) and CT (using a semi-automated approach with 3D Slicer™) components. Both delineations were carried out within previously manually defined “rough” VOIs containing the tumor and the surrounding tissues, which were exploited for the second approach: the same features were extracted from this alternative VOI. Both sets for features were then combined with the clinical variables and processed through the same machine learning (ML) pipelines using the retrospectively recruited patients as the training set and the prospectively recruited patients as the testing set. Logistic regression (LR), random forest (RF), and support vector machine (SVM), as well as their consensus through averaging the output probabilities, were considered for feature selection and modeling for overall survival (OS) prediction as a binary classification (either median OS or 6 months OS). The resulting models were compared in terms of balanced accuracy, sensitivity, and specificity.ResultsOverall, better performance was achieved using the features from delineated tumor volumes. This was observed consistently across ML algorithms and for the two clinical endpoints. However, the loss of performance was not significant, especially when a consensus of the three ML algorithms was considered (0.89 vs. 0.88 and 0.78 vs. 0.77).ConclusionOur findings suggest that it is feasible to achieve similar levels of prognostic accuracy in radiomics-based modeling by relying on a faster and easier VOI definition, skipping a time-consuming tumor delineation step, thus facilitating automation of the whole radiomics workflow. The associated cost is a loss of performance in the resulting models, although this loss can be greatly mitigated when a consensus of several models is relied upon.