Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes

Paula, Mariano De; Ávila, Luis; Martínez, Ernesto

doi:10.1016/j.asoc.2015.06.041

Cited by 28 publications

(21 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, an AI agent can also develop its own experiences through trial-and-error, as humans do [16] and can perform tasks that a classic controller cannot do. Such actions may include selecting an operation strategy, operating nonlinear systems, making decisions based on current conditions, and optimizing operations [17]- [20].…”

Section: A Deep Reinforcement Learningmentioning

confidence: 99%

Algorithm for Autonomous Power-Increase Operation Using Deep Reinforcement Learning and a Rule-Based System

2020

View full text Add to dashboard Cite

The power start-up operation of a nuclear power plant (NPP) increases the reactor power to the full-power condition for electricity generation. Compared to full-power operation, the power-increase operation requires significantly more decision-making and therefore increases the potential for human errors. While previous studies have investigated the use of artificial intelligence (AI) techniques for NPP control, none of them have addressed making the relatively complicated power-increase operation fully autonomous. This study focused on developing an algorithm for converting all the currently manual activities in the NPP power-increase process to autonomous operations. An asynchronous advantage actorcritic, which is a type of deep reinforcement learning method, and a long short-term memory network were applied to the operator tasks for which establishing clear rules or logic was challenging, while a rule-based system was developed for those actions, which could be described by simple logic (such as if-then logic). The proposed autonomous power-increase control algorithm was trained and validated using a compact nuclear simulator (CNS). The simulation results were used to evaluate the algorithm's ability to control the parameters within allowable limits, and the proposed power-increase control algorithm was proven capable of identifying an acceptable operation path for increasing the reactor power from 2% to 100% at a specified rate of power increase. In addition, the pattern of operation that resulted from the autonomous control simulation was found to be identical to that of the established operation strategy. These results demonstrate the potential feasibility of fully autonomous control of the NPP power-increase operation.

show abstract

Section: A Deep Reinforcement Learningmentioning

confidence: 99%

Algorithm for Autonomous Power-Increase Operation Using Deep Reinforcement Learning and a Rule-Based System

2020

View full text Add to dashboard Cite

show abstract

“…TITLE-ABS-KEY( ("reinforcement learning" OR "contextual bandit") AND ("personalization" OR "personalized" OR "personal" OR "personalisation" OR "personalised" OR "customization" OR "customized" OR "customised" OR "customised" OR "individualized" OR "individualised" OR "tailored")) Table 7 Table containing all included publications. The first column refers to the data items in Table 2 # Value Publications 1 n [1,4,10,11,13,16,[18][19][20][24][25][26][27][28][29]31,32,35,36,38,[40][41][42][43][44][45]48,49,52,56,63,66,68,70,74,82,85,86,88,91,93,94,99,101,104,[106]…”

Section: Den Hengst Et Al / Reinforcement Learning For Personalizatimentioning

confidence: 99%

Reinforcement learning for personalization: A systematic literature review

Hengst

Grua

Hassouni

et al. 2020

View full text Add to dashboard Cite

The major application areas of reinforcement learning (RL) have traditionally been game playing and continuous control. In recent years, however, RL has been increasingly applied in systems that interact with humans. RL can personalize digital systems to make them more relevant to individual users. Challenges in personalization settings may be different from challenges found in traditional application areas of RL. An overview of work that uses RL for personalization, however, is lacking. In this work, we introduce a framework of personalization settings and use it in a systematic literature review. Besides setting, we review solutions and evaluation strategies. Results show that RL has been increasingly applied to personalization problems and realistic evaluations have become more prevalent. RL has become sufficiently robust to apply in contexts that involve humans and the field as a whole is growing. However, it seems not to be maturing: the ratios of studies that include a comparison or a realistic evaluation are not showing upward trends and the vast majority of algorithms are used only once. This review can be used to find related work across domains, provides insights into the state of the field and identifies opportunities for future work.

show abstract

“…The predictions of three popular (in the context of glucose prediction) predictors have been used as the input to the DCP: a Feed-forward Neural Network (FFNN), a Gaussian Process regressor (GP), and an Extreme Learning Machine network (ELM) [5], [7]- [10], [14]. All the models have been optimized through the tuning of their hyperparameters.…”

Section: B Base Predictorsmentioning

confidence: 99%

“…Among them, Sun et al proposed a generic predictive model using Long Short-Term Memory (LSTM) and bidirectional LSTM neural networks to predict glucose at prediction horizons (PH) up to 60 minutes [6]. De Paula et al studied the use of Gaussian Processes (GP) to predict future glucose values in an automated glucose controller based on reinforcement learning [7]. Besides, in their work, Georga et al analyzed different types of Extreme Learning Machine (ELM) networks for online shortterm glucose prediction [8].…”

Section: Introductionmentioning

confidence: 99%

Model Fusion to Enhance the Clinical Acceptability of Long-Term Glucose Predictions

Bois

Ammi²,

Yacoubi

2019

2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE)

View full text Add to dashboard Cite

This paper presents the Derivatives Combination Predictor (DCP), a novel model fusion algorithm for making long-term glucose predictions for diabetic people. First, using the history of glucose predictions made by several models, the future glucose variation at a given horizon is predicted. Then, by accumulating the past predicted variations starting from a known glucose value, the fused glucose prediction is computed. A new loss function is introduced to make the DCP model learn to react faster to changes in glucose variations. The algorithm has been tested on 10 in-silico type-1 diabetic children from the T1DMS software. Three initial predictors have been used: a Gaussian process regressor, a feed-forward neural network and an extreme learning machine model. The DCP and two other fusion algorithms have been evaluated at a prediction horizon of 120 minutes with the root-mean-squared error of the prediction, the root-mean-squared error of the predicted variation, and the continuous glucose-error grid analysis. By making a successful trade-off between prediction accuracy and predicted-variation accuracy, the DCP, alongside with its specifically designed loss function, improves the clinical acceptability of the predictions, and therefore the safety of the model for diabetic people.

show abstract

Controlling blood glucose variability under uncertainty using reinforcement learning and Gaussian processes

Cited by 28 publications

References 40 publications

Algorithm for Autonomous Power-Increase Operation Using Deep Reinforcement Learning and a Rule-Based System

Algorithm for Autonomous Power-Increase Operation Using Deep Reinforcement Learning and a Rule-Based System

Reinforcement learning for personalization: A systematic literature review

Model Fusion to Enhance the Clinical Acceptability of Long-Term Glucose Predictions

Contact Info

Product

Resources

About