Personalized Dual-Hormone Control for Type 1 Diabetes Using Deep Reinforcement Learning

Zhu, Taiyu; Li, Kezhi; Georgiou, Pantelis

doi:10.1007/978-3-030-53352-6_5

Cited by 11 publications

(8 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, a dataset with a much larger sample size such as the Tidepool Big Data Donation Dataset [57], may be required to capture the effects of inter-subject variability. Finally, FCNN can be used as an encoder in deep reinforcement learning to extract hidden features from physiological environment and improve decision support systems and automated insulin delivery algorithms (e.g., artificial pancreas) [66]- [68].…”

Section: Discussionmentioning

confidence: 99%

Personalized Blood Glucose Prediction for Type 1 Diabetes Using Evidential Deep Learning and Meta-Learning

Zhu

Herrero

et al. 2023

IEEE Trans. Biomed. Eng.

Self Cite

View full text Add to dashboard Cite

The availability of large amounts of data from continuous glucose monitoring (CGM), together with the latest advances in deep learning techniques, have opened the door to a new paradigm of algorithm design for personalized blood glucose (BG) prediction in type 1 diabetes (T1D) with superior performance. However, there are several challenges that prevent the widespread implementation of deep learning algorithms in actual clinical settings, including unclear prediction confidence and limited training data for new T1D subjects. To this end, we propose a novel deep learning framework, Fast-adaptive and Confident Neural Network (FCNN), to meet these clinical challenges. In particular, an attention-based recurrent neural network is used to learn representations from CGM input and forward a weighted sum of hidden states to an evidential output layer, aiming to compute personalized BG predictions with theoretically supported model confidence. The model-agnostic meta-learning is employed to enable fast adaptation for a new T1D subject with limited training data. The proposed framework has been validated on three clinical datasets.In particular, for a dataset including 12 subjects with T1D, FCNN achieved a root mean square error of 18.64±2.60 mg/dL and 31.07±3.62 mg/dL for 30 and 60-minute prediction horizons, respectively, which outperformed all the considered baseline methods with significant improvements. These results indicate that FCNN is a viable and effective approach for predicting BG levels in T1D. The well-trained models can be implemented in smartphone apps to improve glycemic control by enabling proactive actions through real-time glucose alerts.

show abstract

Section: Discussionmentioning

confidence: 99%

Personalized Blood Glucose Prediction for Type 1 Diabetes Using Evidential Deep Learning and Meta-Learning

Zhu

Herrero

et al. 2023

IEEE Trans. Biomed. Eng.

Self Cite

View full text Add to dashboard Cite

show abstract

“…This state was selected in place of the full sequence of blood glucose, carbohydrate and insulin data utilised in other approaches to avoid modifying the offline RL methods to incorporate recurrency and to reduce state dimensionality [13,12,44]. The reward for the agent was given by the negative of the Magni risk function, which models the clinical risk for a given blood glucose value [22].…”

Section: Problem Formulationmentioning

confidence: 99%

Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes

Emerson¹,

Guy²,

McConville³

2022

Preprint

View full text Add to dashboard Cite

Hybrid closed loop systems represent the future of care for people with type 1 diabetes (T1D). These devices usually utilise simple control algorithms to select the optimal insulin dose for maintaining blood glucose levels within a healthy range. Online reinforcement learning (RL) has been utilised as a method for further enhancing glucose control in these devices. Previous approaches have been shown to reduce patient risk and improve time spent in the target range when compared to classical control algorithms, but are prone to instability in the learning process, often resulting in the selection of unsafe actions. This work presents an evaluation of offline RL as a means for developing clinically effective dosing policies without the need for patient interaction. This paper examines the utility of BCQ, CQL and TD3-BC in managing the blood glucose of nine virtual patients within the UVA/Padova glucose dynamics simulator. When trained on less than a tenth of the data required by online RL approaches, this work shows that offline RL can significantly increase time in the healthy blood glucose range when compared to the strongest state-of-art baseline. This is achieved without any associated increase in low blood glucose events. Offline RL is also shown to be able to correct for common and challenging scenarios such as incorrect bolus dosing, irregular meal timings and sub-optimal training data. The code for this work is available at: https://github.com/hemerson1/offline-glucose.

show abstract

“…As far as difficulty is concerned, HeartPole ( Härmä et al, 2021 ), simglucose ( Zhu et al, 2021 ), and GYMIC ( Kiani et al, 2019 ) are known to be solvable with relatively small models and standard reinforcement learning algorithms like DQN ( Mnih et al, 2015 ). Thus, the only simulators difficult enough to be benchmarks for novel approaches are Virtu-ALS and Auto-ALS and Auto-ALS is the more accessible of the two.…”

Section: Effectivenessmentioning

confidence: 99%

Towards Effective Patient Simulators

Härmä

Petković

2021

Front. Artif. Intell.

View full text Add to dashboard Cite

In this paper we give an overview of the field of patient simulators and provide qualitative and quantitative comparison of different modeling and simulation approaches. Simulators can be used to train human caregivers but also to develop and optimize algorithms for clinical decision support applications and test and validate interventions. In this paper we introduce three novel patient simulators with different levels of representational accuracy: HeartPole, a simplistic transparent rule-based system, GraphSim, a graph-based model trained on intensive care data, and Auto-ALS—an adjusted version of an educational software package used for training junior healthcare professionals. We provide a qualitative and quantitative comparison of the previously existing as well as proposed simulators.

show abstract

Personalized Dual-Hormone Control for Type 1 Diabetes Using Deep Reinforcement Learning

Cited by 11 publications

References 17 publications

Personalized Blood Glucose Prediction for Type 1 Diabetes Using Evidential Deep Learning and Meta-Learning

Personalized Blood Glucose Prediction for Type 1 Diabetes Using Evidential Deep Learning and Meta-Learning

Offline Reinforcement Learning for Safer Blood Glucose Control in People with Type 1 Diabetes

Towards Effective Patient Simulators

Contact Info

Product

Resources

About