GLYFE: review and benchmark of personalized glucose predictive models in type 1 diabetes

Abstract

Due to the sensitive nature of diabetes-related data, preventing them from being shared between studies, progress in the field of glucose prediction is hard to assess. To address this issue, we present GLYFE (GLYcemia Forecasting Evaluation), a benchmark of machine-learning-based glucose-predictive models.To ensure the reproducibility of the results and the usability of the benchmark in the future, we provide extensive details about the data flow. Two datasets are used, the first comprising 10 insilico adults from the UVA/Padova Type 1 Diabetes Metabolic Simulator (T1DMS) and the second being made of 6 real type-1 diabetic patients coming from the OhioT1DM dataset. The predictive models are personalized to the patient and evaluated on 3 different prediction horizons (30, 60, and 120 minutes) with metrics assessing their accuracy and clinical acceptability.The results of nine different models coming from the glucoseprediction literature are presented. First, they show that standard autoregressive linear models are outclassed by kernel-based nonlinear ones and neural networks. In particular, the support vector regression model stands out, being at the same time one of the most accurate and clinically acceptable model. Finally, the relative performances of the models are the same for both datasets. This shows that, even though data simulated by T1DMS are not fully representative of real-world data, they can be used to assess the forecasting ability of the glucose-predictive models.Those results serve as a basis of comparison for future studies. In a field where data are hard to obtain, and where the comparison of results from different studies is often irrelevant, GLYFE gives the opportunity of gathering researchers around a standardized common environment.

Blood glucose forecasting from temporal and static information in children with T1D

“…When we compare the results on our dataset to the literature, we note that we do not reach the performance of comparable models on the OhioT1DM dataset ( 4 ) consisting of much longer time series per participant (8 weeks) and comprising only adults. In a recent benchmark ( 9 ), including deep learning and non-deep learning architectures, an LSTM was among the best-performing methods, reaching an RMSE of around for a prediction horizon (PH) of min and an RMSE of for PH equal to 120 min. 3 Further, the reference measure for the OhioT1DM dataset was reported to be and for PH equal to and , which is much lower than for our dataset—indicating that the prediction task is harder for our dataset.…”

“…To reduce hyper-or hypoglycemic excursions, reliable prediction of future blood glucose levels from previous measurements is desirable for children, as well as adults with T1D. Since the release of the OhioT1DM dataset (4), which consists of data of 6, and later 12 (5) adults with T1D, the topic of blood glucose forecasting has been picked up by the machine learning community (6)(7)(8)(9)(10). For example, McShinsky and Marshall (7) investigated the performance of classical non deep-learning based methods such as autoregressive moving average (ARIMA), random forests, and support vector machines (SVM) for forecasting blood glucose values.…”

Long-Term Prediction of Blood Glucose Levels in Type 1 Diabetes Using a CNN-LSTM-Based Deep Neural Network

“…21 Autoregressive with exogenous inputs is a good reference model since it has been used in many studies in the diabetes literature. Moreover, the choice of SVR models was motivated by the fact that among ML methods SVR has demonstrated promising clinical acceptability, 33 whereas LSTM, designed with the same architecture as the LSTM section of the proposed CNN-LSTM model, allows us to demonstrate the advantage of the added convolutional layers for prediction. In addition, since CRNN, which is comprised of three convolutional and one LSTM layer, and its modifications has shown promising results in various studies, 21,34 we implemented it based on the code repository provided in.…”

Integration of clinical criteria into the training of deep models: Application to glucose prediction for diabetic people

“…As a consequence, the models are trained on maximizing the accuracy of the predictions. However, in the benchmark study we recently conducted [13], we showed that a good accuracy does not ensure that the predictions are clinically acceptable. Indeed, some errors, despite their relatively low magnitude, can be very dangerous for the patient (e.g., errors in the hypoglycemia region).…”