Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study

Abstract

BackgroundSelection of candidates for lung cancer screening based on individual risk has been proposed as an alternative to criteria based on age and cumulative smoking exposure (pack-years). Nine previously established risk models were assessed for their ability to identify those most likely to develop or die from lung cancer. All models considered age and various aspects of smoking exposure (smoking status, smoking duration, cigarettes per day, pack-years smoked, time since smoking cessation) as risk predictors. In addition, some models considered factors such as gender, race, ethnicity, education, body mass index, chronic obstructive pulmonary disease, emphysema, personal history of cancer, personal history of pneumonia, and family history of lung cancer.Methods and findingsRetrospective analyses were performed on 53,452 National Lung Screening Trial (NLST) participants (1,925 lung cancer cases and 884 lung cancer deaths) and 80,672 Prostate, Lung, Colorectal and Ovarian Cancer Screening Trial (PLCO) ever-smoking participants (1,463 lung cancer cases and 915 lung cancer deaths). Six-year lung cancer incidence and mortality risk predictions were assessed for (1) calibration (graphically) by comparing the agreement between the predicted and the observed risks, (2) discrimination (area under the receiver operating characteristic curve [AUC]) between individuals with and without lung cancer (death), and (3) clinical usefulness (net benefit in decision curve analysis) by identifying risk thresholds at which applying risk-based eligibility would improve lung cancer screening efficacy. To further assess performance, risk model sensitivities and specificities in the PLCO were compared to those based on the NLST eligibility criteria. Calibration was satisfactory, but discrimination ranged widely (AUCs from 0.61 to 0.81). The models outperformed the NLST eligibility criteria over a substantial range of risk thresholds in decision curve analysis, with a higher sensitivity for all models and a slightly higher specificity for some models. The PLCOm2012, Bach, and Two-Stage Clonal Expansion incidence models had the best overall performance, with AUCs >0.68 in the NLST and >0.77 in the PLCO. These three models had the highest sensitivity and specificity for predicting 6-y lung cancer incidence in the PLCO chest radiography arm, with sensitivities >79.8% and specificities >62.3%. In contrast, the NLST eligibility criteria yielded a sensitivity of 71.4% and a specificity of 62.2%. Limitations of this study include the lack of identification of optimal risk thresholds, as this requires additional information on the long-term benefits (e.g., life-years gained and mortality reduction) and harms (e.g., overdiagnosis) of risk-based screening strategies using these models. In addition, information on some predictor variables included in the risk prediction models was not available.ConclusionsSelection of individuals for lung cancer screening using individual risk is superior to selection criteria based on age and pack-years alone. T...

Prediction of major depressive disorder onset in college students

“…(WHO) World Mental Health Surveys(Auerbach et al, 2016). Prediction accuracy (AUC = 0.73) was comparable to the few prediction algorithms that have been evaluated for depression within a general population (AUC = 0.71)(Nigatu, Liu, & Wang, 2016) and primary care samples (AUC = 0.82)(Bellon et al, 2011) and are also comparable to other fields of medicine(Karnes et al, 2017;ten Haaf et al, 2017). Our data suggest that the first year in college constitutes a risk period for the onset of MDD.…”

“…Second, our study further adds to the cumulating evidence that the development of risk-prediction for psychiatric disorders is feasible (Bernardini et al, 2017) and provides evidence that a multivariate prediction model can be a useful tool to accurately predict the onset of MDD during college. Prediction accuracy (AUC = 0.73) was comparable to the few prediction algorithms that have been evaluated for depression within a general population (AUC = 0.71) (Nigatu, Liu, & Wang, 2016) and primary care samples (AUC = 0.82) (Bellon et al, 2011) and are also comparable to other fields of medicine (Karnes et al, 2017;ten Haaf et al, 2017). However, to achieve optimal performance, recalibration of models is needed prior to applying the models to a new population.…”

Early detection of sepsis utilizing deep learning on electronic health record event sequences

“…However, while discrimination is an important statistical property, it does not properly address clinical usefulness [15,16,17,27,28,29]. For example, if a false negative decision causes greater harm than a false positive decision, a model with high sensitivity may be preferable to a model with high specificity and lower sensitivity, although the latter model might have, say, a higher AUROC.…”

Applying Risk Prediction Models to Optimize Lung Cancer Screening: Current Knowledge, Challenges, and Future Directions

“…The LLP model also exhibited modest to good discrimination (AUCs, 0.67–0.82) when assessed alone in several European and U.S. study populations [44]. In more recent and extensive external validation studies, however, the PLCO M2012 model demonstrated the best performance, with respect to discrimination, calibration, sensitivity, and specificity, although not exceedingly better than the Bach model [45, 46]. In support, a study of >95,000 Australian smokers aged ≥45 years also found that the PLCO M2012 model displayed good calibration and discrimination (AUC, 0.80), and that its performance was largely driven by the main predictors of the Bach model, age and smoking history [47].…”