Post cardiac surgery recovery process with reinforcement learning

Sandu, Ciprian; Popescu, Dumitru; Popescu, Catalina

doi:10.1109/icstcc.2015.7321368

Cited by 5 publications

(2 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nemati et al developed deep RL algorithms that learn an optimal heparin dosing policy from real trails in large electronic medical records [19, 20]. Sandu et al studied the blood pressure regulation problem in post cardiac surgery patients using RL [21]. Padmanabhan et al resorted to RL for the control of continuous intravenous infusion of propofol for ICU patients by both considering anesthetic effect and regulating the mean arterial pressure to a desired range [8].…”

Section: Related Workmentioning

confidence: 99%

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Liu

Zhao

2019

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Reinforcement learning (RL) provides a promising technique to solve complex sequential decision making problems in health care domains. To ensure such applications, an explicit reward function encoding domain knowledge should be specified beforehand to indicate the goal of tasks. However, there is usually no explicit information regarding the reward function in medical records. It is then necessary to consider an approach whereby the reward function can be learned from a set of presumably optimal treatment trajectories using retrospective real medical data. This paper applies inverse RL in inferring the reward functions that clinicians have in mind during their decisions on weaning of mechanical ventilation and sedative dosing in Intensive Care Units (ICUs). Methods We model the decision making problem as a Markov Decision Process, and use a batch RL method, Fitted Q Iterations with Gradient Boosting Decision Tree , to learn a suitable ventilator weaning policy from real trajectories in retrospective ICU data. A Bayesian inverse RL method is then applied to infer the latent reward functions in terms of weights in trading off various aspects of evaluation criterion. We then evaluate how the policy learned using the Bayesian inverse RL method matches the policy given by clinicians, as compared to other policies learned with fixed reward functions. Results Results show that the inverse RL method is capable of extracting meaningful indicators for recommending extubation readiness and sedative dosage, indicating that clinicians pay more attention to patients’ physiological stability (e.g., heart rate and respiration rate), rather than oxygenation criteria ( FiO 2 , PEEP and SpO 2 ) which is supported by previous RL methods. Moreover, by discovering the optimal weights, new effective treatment protocols can be suggested. Conclusions Inverse RL is an effective approach to discovering clinicians’ underlying reward functions for designing better treatment protocols in the ventilation weaning and sedative dosing in future ICUs.

show abstract

Section: Related Workmentioning

confidence: 99%

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Liu

Zhao

2019

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…One such technique incorporating relative harms based upon clinician judgment includes reinforcement learning, in which a prediction model seeks to learn an optimal individualized treatment plan; currently, early-stage applications of reinforcement learning have been deployed into anesthesiology and critical care settings. 48,49…”

Section: Discussionmentioning

confidence: 99%

Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

et al. 2022

View full text Add to dashboard Cite

Background Postoperative hemodynamic deterioration among cardiac surgical patients can indicate or lead to adverse outcomes. Whereas prediction models for such events using electronic health records or physiologic waveform data are previously described, their combined value remains incompletely defined. The authors hypothesized that models incorporating electronic health record and processed waveform signal data (electrocardiogram lead II, pulse plethysmography, arterial catheter tracing) would yield improved performance versus either modality alone. Methods Intensive care unit data were reviewed following elective adult cardiac surgical procedures at an academic center between 2013-2020. Model features included electronic health record features and physiologic waveforms. Tensor decomposition was used for waveform feature reduction. Machine learning-based prediction models included a 2013-2017 training set and a 2017-2020 temporal holdout test set. The primary outcome was a postoperative deterioration event, defined as a composite of low cardiac index <2.0mL/min/m2, mean arterial pressure <55mmHg sustained ≥120 minutes, new or escalated inotrope/vasopressor infusion, epinephrine bolus ≥1mg, or intensive care unit mortality. Prediction models analyzed data 8 hours prior to events. Results Among 1,555 cases, 185 (12%) experienced 276 deterioration events, most commonly including low cardiac index (7.0% of patients), new inotrope (1.9%), and sustained hypotension (1.4%). The best performing model on the 2013-2017 training set yielded a c-statistic of 0.803 (95% CI 0.799-0.807) although performance was substantially lower in the 2017-2020 test set (0.709, 0.705-0.712). Test set performance of the combined model was greater than corresponding models limited to solely electronic health record features (0.641, 0.637-0.646) or waveform features (0.697, 0.693-0.701). Conclusions Clinical deterioration prediction models combining electronic health record data and waveform data were superior to either modality alone, and performance of combined models was primarily driven by waveform data. Decreased performance of prediction models during temporal validation may be explained by dataset shift, a core challenge of healthcare prediction modeling.

show abstract

Application of Artificial Intelligence in Cardiovascular Medicine

Cheng

Manandhar

Aryal

et al. 2021

Comprehensive Physiology

View full text Add to dashboard Cite

The advent of advances in machine learning (ML)-based techniques has popularized wide applications of artificial intelligence (AI) in various fields ranging from robotics to medicine. In recent years, there has been a surge in the application of AI to research in cardiovascular medicine, which is largely driven by the availability of large-scale clinical and multi-omics datasets. Such applications are providing a new perspective for a better understanding of cardiovascular disease (CVD), which could be used to develop novel diagnostic and therapeutic strategies. For example, studies have shown that ML has a substantial potential for early diagnosis of different types of CVD, prediction of adverse disease outcomes such as heart failure, and development of newer and personalized treatments. In this article, we provide an overview and discuss the current status of a wide range of AI applications, including machine learning, reinforcement learning, and deep learning, in cardiovascular medicine.

show abstract

Post cardiac surgery recovery process with reinforcement learning

Cited by 5 publications

References 3 publications

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Inverse reinforcement learning for intelligent mechanical ventilation and sedative dosing in intensive care units

Prediction of Postoperative Deterioration in Cardiac Surgery Patients Using Electronic Health Record and Physiologic Waveform Data

Application of Artificial Intelligence in Cardiovascular Medicine

Contact Info

Product

Resources

About