Predicting unplanned medical visits among patients with diabetes: translation from machine learning to clinical implementation

Abstract

Background Diabetes is a medical and economic burden in the United States. In this study, a machine learning predictive model was developed to predict unplanned medical visits among patients with diabetes, and findings were used to design a clinical intervention in the sponsoring healthcare organization. This study presents a case study of how predictive analytics can inform clinical actions, and describes practical factors that must be incorporated in order to translate research into clinical practice. Methods Data were drawn from electronic medical records (EMRs) from a large healthcare organization in the Northern Plains region of the US, from adult (≥ 18 years old) patients with type 1 or type 2 diabetes who received care at least once during the 3-year period. A variety of machine-learning classification models were run using standard EMR variables as predictors (age, body mass index (BMI), systolic blood pressure (BP), diastolic BP, low-density lipoprotein, high-density lipoprotein (HDL), glycohemoglobin (A1C), smoking status, number of diagnoses and number of prescriptions). The best-performing model after cross-validation testing was analyzed to identify strongest predictors. Results The best-performing model was a linear-basis support vector machine, which achieved a balanced accuracy (average of sensitivity and specificity) of 65.7%. This model outperformed a conventional logistic regression by 0.4 percentage points. A sensitivity analysis identified BP and HDL as the strongest predictors, such that disrupting these variables with random noise decreased the model’s overall balanced accuracy by 1.3 and 1.4 percentage points, respectively. These recommendations, along with stakeholder engagement, behavioral economics strategies, and implementation science principles helped to inform the design of a clinical intervention targeting behavioral changes. Conclusion Our machine-learning predictive model more accurately predicted unplanned medical visits among patients with diabetes, relative to conventional models. Post-hoc analysis of the model was used for hypothesis generation, namely that HDL and BP are the strongest contributors to unplanned medical visits among patients with diabetes. These findings were translated into a clinical intervention now being piloted at the sponsoring healthcare organization. In this way, this predictive model can be used in moving from prediction to implementation and improved diabetes care management in clinical settings.

A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems

“…Implementation of fraud detection tools [53] to identify anomalies on financial applications using outlier detection techniques such as Local outlier factor, Isolation factor and Elliptic envelope and ML techniques (Random forest, Adaptive boosting and extreme gradient boosting) showed predicted accuracy score of 0.9995. An unplanned modeling [54] for medical visits by patients suffering from diabetes with ML techniques; logistic regression, support vector machine, linear discriminant analysis, quadratic discriminant analysis, extreme gradient boosting, neural networks and deep neural network obtained a balanced accuracy score of 65.7%. Similarly, predicting length of stay [55] from admission to a clinical ward with ML techniques such as random forest, decision trees, support vector machine, multi-layer perceptron, adaboost and gradient boost concluded with random forest as the best performing technique with a balanced accuracy score of 0.72 at the initial stage of admission and 0.75 in-admission.…”

Artificial intelligence to predict bed bath time in Intensive Care Units

“…Using AI tools has been boosted by the need to obtain better results for patients at lower costs from more data ( 2 - 3 ) . Prediction methods based on multiple logistic or linear regressions have been used in different research; however, machine learning models offer the additional possibility of improving prediction based on detecting patterns of many variables simultaneously ( 4 ) . It is believed that, with advances in scientific technical evolution, AI will fundamentally transform healthcare and nursing care.…”

“…Furthermore, there is an important role played by AI in the management of materials and human resources for patient care ( 4 ) . This applicability can be useful for sizing nursing staff, especially in units with high demand for care, such as ICUs, where work overload is a complaint frequently reported by the team and can cause greater absenteeism ( 8 ) .…”

Machine learning clinical decision support for interdisciplinary multimodal chronic musculoskeletal pain treatment

“…There are numerous examples of machine learning being successfully implemented in clinical settings, including for diabetes ( 10 ), prediction of low back pain ( 11 ), improvement of back pain outcomes ( 12 ), self-referral decisions ( 13 ), and self-management of low back pain ( 14 ). Machine learning methods have been used to assess pain diagnosis and prediction of developing chronic pain ( 15 ).…”