Statistical inference for decision curve analysis, with applications to cataract diagnosis

Sande, Sumaiya Z.; Li, Jialiang; D’Agostino, Ralph B.; Wong, Tien Yin; Cheng, Ching‐Yu

doi:10.1002/sim.8588

Cited by 14 publications

(12 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“… 35 Meanwhile, the DCA was used to evaluate the utility of decision models. 36 Compared with the predicted results of the nomogram, the AUC and DCA of RFC were relatively high, which mirrors RFC is a new supervised learning algorithm, at least could be evaluated as lymph node stage and take on a better role than GLM.…”

Section: Discussionmentioning

confidence: 86%

Supervised Learning Based Systemic Inflammatory Markers Enable Accurate Additional Surgery for pT1NxM0 Colorectal Cancer: A Comparative Analysis of Two Practical Prediction Models for Lymph Node Metastasis

Jin¹,

Zhou²,

Sun³

et al. 2021

CMAR

View full text Add to dashboard Cite

Purpose: Predicting lymph node metastasis (LNM) after endoscopic resection is crucial in determining whether patients with pT1NxM0 colorectal cancer (CRC) should undergo additional surgery. This study was aimed to develop a predictive model that can be used to reduce the current likelihood of overtreatment. Patients and Methods: We recruited a total of 1194 consecutive CRC patients with pT1NxM0 who underwent endoscopic or surgical resection at the Gezhouba Central Hospital of Sinopharm between January 1, 2006, and August 31, 2021. The random forest classifier (RFC) and generalized linear algorithm (GLM) were used to screen out the variables that greatly affected the LNM prediction, respectively. The area under the curve (AUC) and decision curve analysis (DCA) were applied to assess the accuracy of predictive models. Results: Analysis identified the top 10 candidate factors including depth of submucosal invasion, neutrophil-lymphocyte ratio (NLR), platelet lymphocyte ratio (PLR), platelet-toneutrophil ratio(PNR), venous invasion, poorly differentiated clusters, tumor budding, grade, lymphatic vascular invasion, and background adenoma. The performance of the GLM achieved the highest AUC of 0.79 (95% confidence interval [CI]: 0.30 to 1.28) in the training cohort and robust AUC of 0.80 (95% confidence interval [CI]: 0.36 to 1.24) in the validation cohort. Meanwhile, the RFC exhibited a robust AUC of 0.84 (95% confidence interval [CI]: 0.40 to 1.28) in the training cohort and a high AUC of 0.85 (95% CI: 0.41 to 1.29) in the validation cohort. DCAs also showed that the RFC had superior predictive ability. Conclusion: Our supervised learning-based model incorporating histopathologic parameters and inflammatory markers showed a more accurate predictive performance compared to the GLM. This newly supervised learning-based predictive model can be used to determine an individually tailored treatment strategy.

show abstract

Section: Discussionmentioning

confidence: 86%

Supervised Learning Based Systemic Inflammatory Markers Enable Accurate Additional Surgery for pT1NxM0 Colorectal Cancer: A Comparative Analysis of Two Practical Prediction Models for Lymph Node Metastasis

Jin¹,

Zhou²,

Sun³

et al. 2021

CMAR

View full text Add to dashboard Cite

show abstract

“…DCA is decision analytic method and incorporates the physician or patient's preferences (Fitzgerald et al, 2015;Van Calster et al, 2013;Sande et al, 2020). We usually inspect a decision curve and look for which strategy leads to the greatest net benefit (i.e., the curve above others).…”

Section: Decisionmentioning

confidence: 99%

“…One can then make an informed decision by examining the net benefit graphs. Inference for net benefit has been established in Sande et al (2020).…”

Section: Introductionmentioning

confidence: 99%

Statistical Learning in Medical Research with Decision Threshold and Accuracy Evaluation

Sande¹,

Seng²,

Li³

et al. 2021

Journal of Data Science

Self Cite

View full text Add to dashboard Cite

Machine learning methods are increasingly applied for medical data analysis to reduce human efforts and improve our understanding of disease propagation. When the data is complicated and unstructured, shallow learning methods may not be suitable or feasible. Deep learning neural networks like multilayer perceptron (MLP) and convolutional neural network (CNN), have been incorporated in medical diagnosis and prognosis for better health care practice. For a binary outcome, these learning methods directly output predicted probabilities for patient's health condition. Investigators still need to consider appropriate decision threshold to split the predicted probabilities into positive and negative regions. We review methods to select the cut-off values, including the relatively automatic methods based on optimization of the ROC curve criteria and also the utility-based methods with a net benefit curve. In particular, decision curve analysis (DCA) is now acknowledged in medical studies as a good complement to the ROC analysis for the purpose of decision making. In this paper, we provide the R code to illustrate how to perform the statistical learning methods, select decision threshold to yield the binary prediction and evaluate the accuracy of the resulting classification. This article will help medical decision makers to understand different classification methods and use them in real world scenario.

show abstract

“…For index test results in continuous scale, receiver operating characteristic (ROC) curve and area under the ROC curve are used to indicate the accuracy 1‐3,7,9,10 . In addition, the use of accuracy measures is not limited to the evaluation of diagnostic tests in medical research, but is also relevant to evaluate the predictive performance of machine (or statistical) learning methods 11‐14 …”

Section: Introductionmentioning

confidence: 99%

“…[1][2][3]7,9,10 In addition, the use of accuracy measures is not limited to the evaluation of diagnostic tests in medical research, but is also relevant to evaluate the predictive performance of machine (or statistical) learning methods. [11][12][13][14] It is essential to obtain valid estimates of accuracy measures to ensure the clinical validity of the tests. 7 Biased estimates are misleading, which may result in the premature implementation of the new tests and lead to wrong decision making by the clinicians.…”

Section: Introductionmentioning

confidence: 99%

Correcting for partial verification bias in diagnostic accuracy studies: A tutorial using R

Arifin

Yusof

2022

Statistics in Medicine

View full text Add to dashboard Cite

Diagnostic tests play a crucial role in medical care. Thus any new diagnostic tests must undergo a thorough evaluation. New diagnostic tests are evaluated in comparison with the respective gold standard tests. The performance of binary diagnostic tests is quantified by accuracy measures, with sensitivity and specificity being the most important measures. In any diagnostic accuracy study, the estimates of these measures are often biased owing to selective verification of the patients, which is referred to as partial verification bias. Several methods for correcting partial verification bias are available depending on the scale of the index test, target outcome, and missing data mechanism. However, these are not easily accessible to the researchers due to the complexity of the methods. This article aims to provide a brief overview of the methods available to correct for partial verification bias involving a binary diagnostic test and provide a practical tutorial on how to implement the methods using the statistical programming language R.

show abstract

Statistical inference for decision curve analysis, with applications to cataract diagnosis

Cited by 14 publications

References 53 publications

Supervised Learning Based Systemic Inflammatory Markers Enable Accurate Additional Surgery for pT1NxM0 Colorectal Cancer: A Comparative Analysis of Two Practical Prediction Models for Lymph Node Metastasis

Supervised Learning Based Systemic Inflammatory Markers Enable Accurate Additional Surgery for pT1NxM0 Colorectal Cancer: A Comparative Analysis of Two Practical Prediction Models for Lymph Node Metastasis

Statistical Learning in Medical Research with Decision Threshold and Accuracy Evaluation

Correcting for partial verification bias in diagnostic accuracy studies: A tutorial using R

Contact Info

Product

Resources

About