Automatic control of cardiac ablation catheter with deep reinforcement learning method

You, Hyeonseok; Bae, Eun-Kyung; Moon, Youngjin; Kweon, Jihoon; Choi, Jaesoon

doi:10.1007/s12206-019-1036-0

Cited by 24 publications

(29 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The positioning accuracy (>94%) in simulation and the real environment, even with the external load, indicated that the control approach was effective and robust. You et al (2019) introduced a control strategy enabling the soft catheter to move in a heart model using Dueling DQN (DDQN). This algorithm defines the Q-value as the summation of the state value and the advantage function.…”

Section: Reinforcement Learning Without Kinematics/ Dynamics Modelmentioning

confidence: 99%

“…In the gradient-based approach, the policy function is maximized with gradient-descent iteratively (Thuruthel et al, 2018;Liu et al, 2020). In contrast to policy search reinforcement learning, valuebased methods generate the optimal control policy by optimizing the value function, including SARSA (Ansari et al, 2017b), Q-learning (You et al, 2017;Jiang et al, 2021), DQN (Satheeshbabu et al, 2019;Wu et al, 2020) and its various extensions (e.g., DDQN (You et al, 2019) and Double DQN). The actor-critic approach is a combination of policy-based and value-based reinforcement learning, where the actor executes referring to the policy; thereby the critic calculates the value function to evaluate the actor (Satheeshbabu et al, 2020).…”

Section: Policy-based Vs Value-based Reinforcement Learningmentioning

confidence: 99%

“…This is common in the control for flexible manipulators with high-dimensional action and state variables. Deep reinforcement learning is capable of processing more complicated input formats including images (You et al, 2019) while storing more states and actions, which is particularly significant for continuous control.…”

Section: Policy-based Vs Value-based Reinforcement Learningmentioning

confidence: 99%

See 2 more Smart Citations

A Survey for Machine Learning-Based Control of Continuum Robots

Wang

Kwok

2021

Front. Robot. AI

View full text Add to dashboard Cite

Soft continuum robots have been accepted as a promising category of biomedical robots, accredited to the robots’ inherent compliance that makes them safely interact with their surroundings. In its application of minimally invasive surgery, such a continuum concept shares the same view of robotization for conventional endoscopy/laparoscopy. Different from rigid-link robots with accurate analytical kinematics/dynamics, soft robots encounter modeling uncertainties due to intrinsic and extrinsic factors, which would deteriorate the model-based control performances. However, the trade-off between flexibility and controllability of soft manipulators may not be readily optimized but would be demanded for specific kinds of modeling approaches. To this end, data-driven modeling strategies making use of machine learning algorithms would be an encouraging way out for the control of soft continuum robots. In this article, we attempt to overview the current state of kinematic/dynamic model-free control schemes for continuum manipulators, particularly by learning-based means, and discuss their similarities and differences. Perspectives and trends in the development of new control methods are also investigated through the review of existing limitations and challenges.

show abstract

Section: Reinforcement Learning Without Kinematics/ Dynamics Modelmentioning

confidence: 99%

Section: Policy-based Vs Value-based Reinforcement Learningmentioning

confidence: 99%

Section: Policy-based Vs Value-based Reinforcement Learningmentioning

confidence: 99%

See 1 more Smart Citation

A Survey for Machine Learning-Based Control of Continuum Robots

Wang

Kwok

2021

Front. Robot. AI

View full text Add to dashboard Cite

show abstract

“…In interventional cardiology, reinforcement learning can provide tools to optimize sequences of decisions for long-term outcomes such as improving ST-segment elevation myocardial infarction outcomes or reducing errors in ECG diagnosis. Optimization of treatment policies, real-time decisions and robot navigation are some other applications of reinforcement learning ( 31 , 32 ).…”

Section: Machine and Deep Learning Overviewmentioning

confidence: 99%

Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble

Ali

Pesaranghader

Avram

et al. 2021

Front. Cardiovasc. Med.

View full text Add to dashboard Cite

Driven by recent innovations and technological progress, the increasing quality and amount of biomedical data coupled with the advances in computing power allowed for much progress in artificial intelligence (AI) approaches for health and biomedical research. In interventional cardiology, the hope is for AI to provide automated analysis and deeper interpretation of data from electrocardiography, computed tomography, magnetic resonance imaging, and electronic health records, among others. Furthermore, high-performance predictive models supporting decision-making hold the potential to improve safety, diagnostic and prognostic prediction in patients undergoing interventional cardiology procedures. These applications include robotic-assisted percutaneous coronary intervention procedures and automatic assessment of coronary stenosis during diagnostic coronary angiograms. Machine learning (ML) has been used in these innovations that have improved the field of interventional cardiology, and more recently, deep Learning (DL) has emerged as one of the most successful branches of ML in many applications. It remains to be seen if DL approaches will have a major impact on current and future practice. DL-based predictive systems also have several limitations, including lack of interpretability and lack of generalizability due to cohort heterogeneity and low sample sizes. There are also challenges for the clinical implementation of these systems, such as ethical limits and data privacy. This review is intended to bring the attention of health practitioners and interventional cardiologists to the broad and helpful applications of ML and DL algorithms to date in the field. Their implementation challenges in daily practice and future applications in the field of interventional cardiology are also discussed.

show abstract

“…Implementing deep learning architectures with reinforcement learning algorithms is capable of scaling to previously unsolvable problems (86). In a recent work by You et al, a robot was developed to reduce the radiation exposure of personnel during an interventional procedure for arrhythmia (27). Experiments on the control of an electrophysiology catheter by robots were conducted.…”

Section: Shallow Neuralmentioning

confidence: 99%

Implementing Machine Learning in Interventional Cardiology: The Benefits are Worth the Trouble

Ali¹,

Pesaranghader²,

Avram³

et al. 2021

Preprint

View full text Add to dashboard Cite

Driven by recent innovations and technological progress, the increasing quality and amount of biomedical data coupled with the advances in computing power allowed for much progress in artificial intelligence (AI) approaches for health and biomedical research. In interventional cardiology, the hope is for AI to provide automated analysis and deeper interpretation of data from electrocardiography, computed tomography, magnetic resonance imaging, and electronic health records, among others. Furthermore, high-performance predictive models supporting decision-making hold the potential to improve safety, diagnostic and prognostic prediction in patients undergoing interventional cardiology procedures. These applications include robotic-assisted percutaneous coronary intervention procedures and automatic assessment of coronary stenosis during diagnostic coronary angiograms. Machine learning (ML) has been used in these innovations that have improved the field of interventional cardiology, and more recently, deep learning (DL) has emerged as one of the most successful branches of ML in many applications. It remains to be seen if DL approaches will have a major impact on current and future practice. DL-based predictive systems also have several limitations, including lack of interpretability and lack of generalizability due to cohort heterogeneity and low sample sizes. There are also challenges for the clinical implementation of these systems, such as ethical limits and data privacy. This review is intended to bring the attention of health practitioners and interventional cardiologists to the broad and helpful applications of ML and DL algorithms to date in the field. Their implementation challenges in daily practice and future applications in the field of interventional cardiology are also discussed.

show abstract

Automatic control of cardiac ablation catheter with deep reinforcement learning method

Cited by 24 publications

References 22 publications

A Survey for Machine Learning-Based Control of Continuum Robots

A Survey for Machine Learning-Based Control of Continuum Robots

Implementing Machine Learning in Interventional Cardiology: The Benefits Are Worth the Trouble

Implementing Machine Learning in Interventional Cardiology: The Benefits are Worth the Trouble

Contact Info

Product

Resources

About