Background Diabetes management is complex, and program personalization has been identified to enhance engagement and clinical outcomes in diabetes management programs. However, 50% of individuals living with diabetes are unable to achieve glycemic control, presenting a gap in the delivery of self-management education and behavior change. Machine learning and recommender systems, which have been used within the health care setting, could be a feasible application for diabetes management programs to provide a personalized user experience and improve user engagement and outcomes. Objective This study aims to evaluate machine learning models using member-level engagements to predict improvement in estimated A1c and develop personalized action recommendations within a remote diabetes monitoring program to improve clinical outcomes. Methods A retrospective study of Livongo for Diabetes member engagement data was analyzed within five action categories (interacting with a coach, reading education content, self-monitoring blood glucose level, tracking physical activity, and monitoring nutrition) to build a member-level model to predict if a specific type and level of engagement could lead to improved estimated A1c for members with type 2 diabetes. Engagement and improvement in estimated A1c can be correlated; therefore, the doubly robust learning method was used to model the heterogeneous treatment effect of action engagement on improvements in estimated A1c. Results The treatment effect was successfully computed within the five action categories on estimated A1c reduction for each member. Results show interaction with coaches and self-monitoring blood glucose levels were the actions that resulted in the highest average decrease in estimated A1c (1.7% and 1.4%, respectively) and were the most recommended actions for 54% of the population. However, these were found to not be the optimal interventions for all members; 46% of members were predicted to have better outcomes with one of the other three interventions. Members who engaged with their recommended actions had on average a 0.8% larger reduction in estimated A1c than those who did not engage in recommended actions within the first 3 months of the program. Conclusions Personalized action recommendations using heterogeneous treatment effects to compute the impact of member actions can reduce estimated A1c and be a valuable tool for diabetes management programs in encouraging members toward actions to improve clinical outcomes.

BACKGROUND Diabetes management is complex, and program personalization has been identified to enhance engagement and clinical outcomes in diabetes management programs. However, 50% of individuals living with diabetes are unable to achieve glycemic control, presenting a gap in the delivery of self-management education and behavior change. Machine learning and recommender systems, which have been used within the health care setting, could be a feasible application for diabetes management programs to provide a personalized user experience and improve user engagement and outcomes. OBJECTIVE This study aims to evaluate machine learning models using member-level engagements to predict improvement in estimated A_1c and develop personalized action recommendations within a remote diabetes monitoring program to improve clinical outcomes. METHODS A retrospective study of Livongo for Diabetes member engagement data was analyzed within five action categories (interacting with a coach, reading education content, self-monitoring blood glucose level, tracking physical activity, and monitoring nutrition) to build a member-level model to predict if a specific type and level of engagement could lead to improved estimated A_1c for members with type 2 diabetes. Engagement and improvement in estimated A_1c can be correlated; therefore, the doubly robust learning method was used to model the heterogeneous treatment effect of action engagement on improvements in estimated A_1c. RESULTS The treatment effect was successfully computed within the five action categories on estimated A_1c reduction for each member. Results show interaction with coaches and self-monitoring blood glucose levels were the actions that resulted in the highest average decrease in estimated A_1c (1.7% and 1.4%, respectively) and were the most recommended actions for 54% of the population. However, these were found to not be the optimal interventions for all members; 46% of members were predicted to have better outcomes with one of the other three interventions. Members who engaged with their recommended actions had on average a 0.8% larger reduction in estimated A_1c than those who did not engage in recommended actions within the first 3 months of the program. CONCLUSIONS Personalized action recommendations using heterogeneous treatment effects to compute the impact of member actions can reduce estimated A_1c and be a valuable tool for diabetes management programs in encouraging members toward actions to improve clinical outcomes.