Using a cohort study of diabetes and peripheral artery disease to compare logistic regression and machine learning via random forest modeling

Abstract

Background This study illustrates the use of logistic regression and machine learning methods, specifically random forest models, in health services research by analyzing outcomes for a cohort of patients with concomitant peripheral artery disease and diabetes mellitus. Methods Cohort study using fee-for-service Medicare beneficiaries in 2015 who were newly diagnosed with peripheral artery disease and diabetes mellitus. Exposure variables include whether patients received preventive measures in the 6 months following their index date: HbA1c test, foot exam, or vascular imaging study. Outcomes include any reintervention, lower extremity amputation, and death. We fit both logistic regression models as well as random forest models. Results There were 88,898 fee-for-service Medicare beneficiaries diagnosed with peripheral artery disease and diabetes mellitus in our cohort. The rate of preventative treatments in the first six months following diagnosis were 52% (n = 45,971) with foot exams, 43% (n = 38,393) had vascular imaging, and 50% (n = 44,181) had an HbA1c test. The directionality of the influence for all covariates considered matched those results found with the random forest and logistic regression models. The most predictive covariate in each approach differs as determined by the t-statistics from logistic regression and variable importance (VI) in the random forest model. For amputation we see age 85 + (t = 53.17) urban-residing (VI = 83.42), and for death (t = 65.84, VI = 88.76) and reintervention (t = 34.40, VI = 81.22) both models indicate age is most predictive. Conclusions The use of random forest models to analyze data and provide predictions for patients holds great potential in identifying modifiable patient-level and health-system factors and cohorts for increased surveillance and intervention to improve outcomes for patients. Random forests are incredibly high performing models with difficult interpretation most ideally suited for times when accurate prediction is most desirable and can be used in tandem with more common approaches to provide a more thorough analysis of observational data.

A Non-Invasive Hemoglobin Detection Device Based on Multispectral Photoplethysmography

“…RF improves the stability and accuracy of predictions by combining predictions from multiple trees [ 28 ]. Random Forest can quantify the importance of features and guide feature selection [ 29 ]. The number of estimators was chosen such that the best value in the range of 1–10 and the maximum depth in the range of 1–6 would be found.…”

Predicting amputation using machine learning: A systematic review

“…For simplicity of reporting, the included studies were categorized by amputation etiology. Most studies reported on patients who received amputation due to Diabetes [ 25 – 29 , 35 , 36 , 39 – 41 ], followed by Trauma [ 30 – 32 ], and “Other” [ 33 , 34 ]. All included studies were derivation studies that included a form of validation.…”

“…The variables that were found to be important features in models varied. However, some of these variables, including increased age, Wagner scores, C-reactive protein and history of amputation among others, appeared in multiple models [ 26 – 29 , 36 , 39 – 41 ]. The models within these studies ranged in performance from sub-optimal to excellent [AUC: 0.6–0.94].…”

“…The models within these studies ranged in performance from sub-optimal to excellent [AUC: 0.6–0.94]. Random Forest models [ 25 , 28 , 29 , 35 , 36 ], Gradient Boosted [ 27 , 28 ], and Logistic Regression [ 25 , 29 , 35 , 39 ] were used in multiple studies as the modeling technique. Only two of the studies had comparison reference tests to a non-ML prediction model [ 27 , 40 ].…”

“…The performance metric in the majority of included articles [25][26][27][28][29][30][31][32][33][34][35][36] was the area under the curve of the receiver operator characteristic (AUC) which is standard for the evaluation of application of ML in medical contexts [37,38]. However, three studies used other performance metrics, including F-score (Fβ) [31], out of bag error rate [39], or only accuracy [40]. For simplicity of reporting, the included studies were categorized by amputation etiology.…”