Reinforcement learning-based control for combined infusion of sedatives and analgesics

Padmanabhan, Regina; Meskin, Nader; Haddad, Wassim M.

doi:10.1109/codit.2017.8102643

Cited by 7 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The target for the RL agent was to infuse propofol so that the target BIS would be reached in a short time, whereas MAP was kept within a desired range. In subsequent studies, Padmanabhan et al [ 31 , 32 ] modified their methods with different RL training algorithms (Q-learning and policy iteration). In all their studies, the RL agent was able to suggest accurate propofol doses and achieve target BIS values within a few minutes.…”

Section: Resultsmentioning

confidence: 99%

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review

Liu¹,

See²,

Ngiam³

et al. 2020

J Med Internet Res

136

View full text Add to dashboard Cite

Background Decision support systems based on reinforcement learning (RL) have been implemented to facilitate the delivery of personalized care. This paper aimed to provide a comprehensive review of RL applications in the critical care setting. Objective This review aimed to survey the literature on RL applications for clinical decision support in critical care and to provide insight into the challenges of applying various RL models. Methods We performed an extensive search of the following databases: PubMed, Google Scholar, Institute of Electrical and Electronics Engineers (IEEE), ScienceDirect, Web of Science, Medical Literature Analysis and Retrieval System Online (MEDLINE), and Excerpta Medica Database (EMBASE). Studies published over the past 10 years (2010-2019) that have applied RL for critical care were included. Results We included 21 papers and found that RL has been used to optimize the choice of medications, drug dosing, and timing of interventions and to target personalized laboratory values. We further compared and contrasted the design of the RL models and the evaluation metrics for each application. Conclusions RL has great potential for enhancing decision making in critical care. Challenges regarding RL system design, evaluation metrics, and model choice exist. More importantly, further work is required to validate RL in authentic clinical environments.

show abstract

Section: Resultsmentioning

confidence: 99%

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review

Liu¹,

See²,

Ngiam³

et al. 2020

J Med Internet Res

136

View full text Add to dashboard Cite

show abstract

“…Among the studies that used IPTW (n=36), 17 applied stabilized weights, one applied weight truncation, and eight studies applied both weight stabilization and truncation. Among studies that applied RL on real (ie, not simulated) patient data (n=16), seven studies used an importance-sampling based 34 , model-based 35,36 , a doubly robust OPE method 37 , or a combination of these. Eight studies used the so-called 'U-curve method' 38 (panel 1) and for six of these, this was the only reported OPE method.…”

Section: Method-specific Itemsmentioning

confidence: 99%

Causal inference using observational intensive care unit data: a systematic review and recommendations for future practice

Smit

Krijthe

Bommel

et al. 2022

Preprint

View full text Add to dashboard Cite

Aim: To review and appraise the quality of studies that present models for causal inference of time-varying treatment effects in the adult intensive care unit (ICU) and give recommendations to improve future research practice. Data sources: Embase, MEDLINE ALL, Web of Science Core Collection, Google Scholar, medRxiv, and bioRxiv up to March 2, 2022. Study selection: Studies that present models for causal inference that deal with time-varying treatments in adult ICU patients. Data extraction: From the included studies, data was extracted about the study setting and applied methodology. Quality of reporting (QOR) of target trial components and causal assumptions (ie, conditional exchangeability, positivity and consistency) were assessed. Results: 1,714 titles were screened and 60 studies were included, of which 36 (60%) were published in the last 5 years. G methods were the most commonly used (n=40/60, 67%), further divided into inverse-probability-of-treatment weighting (n=36/40, 90%) and the parametric G formula (n=4/40, 10%). The remaining studies (n=20/60, 33%) used reinforcement learning methods. Overall, most studies (n=36/60, 60%) considered static treatment regimes. Only ten (17%) studies fully reported all five target trial components (ie, eligibility criteria, treatment strategies, follow-up period, outcome and analysis plan). The treatment strategies and analysis plan components were not (fully) reported in 38% and 48% of the studies, respectively. The causal assumptions (ie, conditional exchangeability, positivity and consistency) remained unmentioned in 35%, 68% and 88% of the studies, respectively. All three causal assumptions were mentioned (or a check for potential violations was reported) in only six (10%) studies. Sixteen studies (27%) estimated the treatment effect both by adjusting for baseline confounding and by adjusting for baseline and treatment-affected time-varying confounding, which often led to substantial changes in treatment effect estimates. Conclusions: Studies that present models for causal inference in the ICU were found to have incomplete or missing reporting of target trial components and causal assumptions. To achieve actionable artificial intelligence in the ICU, we advocate careful consideration of the causal question of interest, the use of target trial emulation, usage of appropriate causal inference methods and acknowledgement (and ideally examination of potential violations) of the causal assumptions. Systematic review registration: PROSPERO (CRD42022324014)

show abstract

“…Work [14] proposes a positive control law allowing real-time tuning of the propofol-remifentanil balance while ensuring stability. In [15], a Reinforcement Learning method has been used to address the challenge of the MISO system control design with simulation testing. The authors of [16] put forward a mid-range controller strategy that leverages the use of remifentanil for short-term and small-scale modulation of the bispectral index (BIS), while relying on propofol for longer-term interventions.…”

Section: Introductionmentioning

confidence: 99%

Data-based Pharmacodynamic Modeling for BIS and Mean Arterial Pressure Prediction during General Anesthesia

Aubouin--Pairault

Fiacchini

Dang

2023

2023 European Control Conference (ECC)

View full text Add to dashboard Cite

In this paper, a multi-model predictive control approach is used to automate the co-administration of propofol and remifentanil from bispectral index measurement during general anesthesia. To handle the parameter uncertainties in the non-linear output function, multiple Extended Kalman Filters are used to estimate the state of the system in parallel. The best model is chosen using a model-matching criterion and used in a non-linear MPC to compute the next drug rates. The method is compared with a conventional non-linear MPC approach and a PID from the literature. The robustness of the controller is evaluated using Monte-Carlo simulations on a wide population introducing uncertainties in the models. Both simulation setup and controller codes are accessible in open source for further use. Our preliminary results show the potential interest in using a multi-model method to handle parameter uncertainties.

show abstract

Reinforcement learning-based control for combined infusion of sedatives and analgesics

Cited by 7 publications

References 16 publications

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review

Reinforcement Learning for Clinical Decision Support in Critical Care: Comprehensive Review

Causal inference using observational intensive care unit data: a systematic review and recommendations for future practice

Data-based Pharmacodynamic Modeling for BIS and Mean Arterial Pressure Prediction during General Anesthesia

Contact Info

Product

Resources

About