Development and Validation of Machine Learning–based Model for the Prediction of Malignancy in Multiple Pulmonary Nodules: Analysis from Multicentric Cohorts

Abstract

Purpose: Nodule evaluation is challenging and critical to diagnose multiple pulmonary nodules (MPNs). We aimed to develop and validate a machine learning–based model to estimate the malignant probability of MPNs to guide decision-making. Experimental Design: A boosted ensemble algorithm (XGBoost) was used to predict malignancy using the clinicoradiologic variables of 1,739 nodules from 520 patients with MPNs at a Chinese center. The model (PKU-M model) was trained using 10-fold cross-validation in which hyperparameters were selected and fine-tuned. The model was validated and compared with solitary pulmonary nodule (SPN) models, clinicians, and a computer-aided diagnosis (CADx) system in an independent transnational cohort and a prospective multicentric cohort. Results: The PKU-M model showed excellent discrimination [area under the curve; AUC (95% confidence interval (95% CI)), 0.909 (0.854–0.946)] and calibration (Brier score, 0.122) in the development cohort. External validation (583 nodules) revealed that the AUC of the PKU-M model was 0.890 (0.859–0.916), higher than those of the Brock model [0.806 (0.771–0.838)], PKU model [0.780 (0.743–0.817)], Mayo model [0.739 (0.697–0.776)], and VA model [0.682 (0.640–0.722)]. Prospective comparison (200 nodules) showed that the AUC of the PKU-M model [0.871 (0.815–0.915)] was higher than that of surgeons [0.790 (0.711–0.852), 0.741 (0.662–0.804), and 0.727 (0.650–0.788)], radiologist [0.748 (0.671–0.814)], and the CADx system [0.757 (0.682–0.818)]. Furthermore, the model outperformed the clinicians with an increase of 14.3% in sensitivity and 7.8% in specificity. Conclusions: After its development using machine learning algorithms, validation using transnational multicentric cohorts, and prospective comparison with clinicians and the CADx system, this novel prediction model for MPNs presented solid performance as a convenient reference to help decision-making.

Comprehensive Analysis of Clinical Logistic and Machine Learning-Based Models for the Evaluation of Pulmonary Nodules

“…Most often used and externally validated models (Brock, Mayo, PKU, VA) were selected for a network meta-analysis; the summary receiver operating characteristic (SROC) curve was plotted with the method proposed by Reitsma et al. 15 ; and the area under the SROC curve (AUSROC) was calculated. Sensitivity and specificity of each model were also pooled using analysis of variance model, 16 and diagnostic OR and superiority index were calculated.…”

Development and validation of a nomogram model for lung cancer based on radiomics artificial intelligence score and clinical blood test data

“…With the development of artificial intelligence technology, the machine learning models provided a better alternative for creating applicable predictive clinical diagnosis tools. In this study, we developed and validated a diagnostic nomogram model to improve the diagnostic accuracy of lung cancer based on AI tools and clinical data ( 3 , 10 , 13 ).…”

“…Low-dose computed tomography (LDCT) is the main method for public physical screening. The tumor markers assessment in hospital including carcinoembryonic antigen (CEA) and cytokeratin 19 fragment antigen21-1(CYFRA21-1) can improve the diagnosis rate ( 3 ). Artificial intelligence ( AI) models are a step forward from automated nodule diagnosis, as they typically do not require nodule measurement or data entry.…”

A pairwise radiomics algorithm - lesion pair relation estimation (PRE) model for distinguishing multiple primary lung cancer (MPLC) from intrapulmonary metastasis (IPM)

“…In recent years, there were studies differentiating benign from malignant pulmonary nodules [38,39] or predicting the invasiveness of a lesion [13,40,41] with the help of arti cial intelligence (AI). Machine learning-based models were built for multiple nodules to predict lung malignancy [42], or borrowed from solitary nodules to diagnose MPLC [14]. These methods could be inappropriate in that each patient should be considered as a whole rather than targeting each lesion separately.…”