Machine Learning Approach to Decision Making for Insulin Initiation in Japanese Patients With Type 2 Diabetes (JDDM 58): Model Development and Validation Study

Abstract

Background Applications of machine learning for the early detection of diseases for which a clear-cut diagnostic gold standard exists have been evaluated. However, little is known about the usefulness of machine learning approaches in the decision-making process for decisions such as insulin initiation by diabetes specialists for which no absolute standards exist in clinical settings. Objective The objectives of this study were to examine the ability of machine learning models to predict insulin initiation by specialists and whether the machine learning approach could support decision making by general physicians for insulin initiation in patients with type 2 diabetes. Methods Data from patients prescribed hypoglycemic agents from December 2009 to March 2015 were extracted from diabetes specialists’ registries, resulting in a sample size of 4860 patients who had received initial monotherapy with either insulin (n=293) or noninsulin (n=4567). Neural network output was insulin initiation ranging from 0 to 1 with a cutoff of >0.5 for the dichotomous classification. Accuracy, recall, and area under the receiver operating characteristic curve (AUC) were calculated to compare the ability of machine learning models to make decisions regarding insulin initiation to the decision-making ability of logistic regression and general physicians. By comparing the decision-making ability of machine learning and logistic regression to that of general physicians, 7 cases were chosen based on patient information as the gold standard based on the agreement of 8 of the 9 specialists. Results The AUCs, accuracy, and recall of logistic regression were higher than those of machine learning (AUCs of 0.89-0.90 for logistic regression versus 0.67-0.74 for machine learning). When the examination was limited to cases receiving insulin, discrimination by machine learning was similar to that of logistic regression analysis (recall of 0.05-0.68 for logistic regression versus 0.11-0.52 for machine learning). Accuracies of logistic regression, a machine learning model (downsampling ratio of 1:8), and general physicians were 0.80, 0.70, and 0.66, respectively, for 43 randomly selected cases. For the 7 gold standard cases, the accuracies of logistic regression and the machine learning model were 1.00 and 0.86, respectively, with a downsampling ratio of 1:8, which were higher than the accuracy of general physicians (ie, 0.43). Conclusions Although we found no superior performance of machine learning over logistic regression, machine learning had higher accuracy in prediction of insulin initiation than general physicians, defined by diabetes specialists’ choice of the gold standard. Further study is needed before the use of machine learning–based decision support systems for insulin initiation can be incorporated into clinical practice.

Potential applications and performance of machine learning techniques and algorithms in clinical practice: A systematic review

“…Del Parigi et al [56] showed that ML (Random Forest) could be used to determine the predictors of the highest improvement in glycemic control for an individual patient from the data pooled from randomized controlled trials. In addition, Fujihara et al [81] have shown that ML (Neural Networks) can aid clinicians to decide, with high prediction accuracy, when to initiate an insulin regimen for their patients with Type 2 diabetes. Crutzen et al [82] employed a collection of algorithms to identify patients with type 2 diabetes at an increased risk of hypoglycaemia in primary care.…”

Predicting Antituberculosis Drug–Induced Liver Injury Using an Interpretable Machine Learning Method: Model Development and Validation Study

“…Various models are used to identify drugs associated with the risk of DILI at the preclinical stage [28]. Machine learning models have demonstrated strong predictive power and retained a simple form for communication with researchers [29][30][31][32][33][34][35][36][37][38][39]. XGBoost is a boosting ensemble machine learning algorithm that integrates a few classification and regression trees models to form a strong classifier [40,41].…”

Machine Learning Approach to Drug Treatment Strategy for Diabetes Care

“…However, approximately 35% to 40% of patients worldwide initiating the use of an oral antihyperglycemic drugs did not receive the recommended initial therapy [49][50][51]. Table 1 summarizes studies of the predictability of the use of antihyperglycemic medications using artificial intelligence [52][53][54][55][56][57]. Liu et al [52] investigated prescriptions in 82 patients and reported that 80.2% of recommendations generated from guidelines coincided with the medication classes (metformin, insulin secretagogues or α-glucosidase inhibitors, thiazolidinediones, dipeptidyl peptidase-4 inhibitors [DPP-4I], insulin) from their real prescriptions by K-Nearest Neighbors.…”

“…We recently elucidated the ability of machine learning models and determined whether artificial intelligence might assist clinicians in deciding on the initial insulin therapy for type 2 diabetes mellitus in clinical practice. We recruited 4,860 participants who received initial monotherapy by diabetes specialists [ 56 , 58 ]. We found no superiority of performance of machine learning over logistic regression.…”