Deep Reinforcement Learning for Guidewire Navigation in Coronary Artery Phantom

Kweon, Jihoon; Kim, Kyung-Hwan; Lee, Chaehyuk; Kwon, Hwi; Park, Jin-Woo; Song, Kyoseok; Kim, Young In; Park, Jeeone; Back, Inwook; Roh, Jae‐Hyung; Moon, Youngjin; Choi, Jaesoon; Kim, Young‐Hak

doi:10.1109/access.2021.3135277

Cited by 28 publications

(36 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model evaluated via in-silico experiments achieved high tool-position accuracy with an RMS error of 0.003 mm. An advanced strategy published by Kweon et al [ 71 ] shows that imitation or reinforcement learning can be directly designed for autonomous navigation of endovascular tools. In addition, Karstensen et al [ 72 ] adopted deep deterministic policy gradients with hindsight experience replay for a learning-based control of guidewire navigation in a robot-assisted peripheral vascular-intervention study.…”

Section: Key Technologies and Application Areas Of Vascular Intervent...mentioning

confidence: 99%

Technical and Clinical Progress on Robot-Assisted Endovascular Interventions: A Review

Duan

Akinyemi

et al. 2023

Micromachines

View full text Add to dashboard Cite

Prior methods of patient care have changed in recent years due to the availability of minimally invasive surgical platforms for endovascular interventions. These platforms have demonstrated the ability to improve patients’ vascular intervention outcomes, and global morbidities and mortalities from vascular disease are decreasing. Nonetheless, there are still concerns about the long-term effects of exposing interventionalists and patients to the operational hazards in the cath lab, and the perioperative risks that patients undergo. For these reasons, robot-assisted vascular interventions were developed to provide interventionalists with the ability to perform minimally invasive procedures with improved surgical workflow. We conducted a thorough literature search and presented a review of 130 studies published within the last 20 years that focused on robot-assisted endovascular interventions and are closely related to the current gains and obstacles of vascular interventional robots published up to 2022. We assessed both the research-based prototypes and commercial products, with an emphasis on their technical characteristics and application domains. Furthermore, we outlined how the robotic platforms enhanced both surgeons’ and patients’ perioperative experiences of robot-assisted vascular interventions. Finally, we summarized our findings and proposed three key milestones that could improve the development of the next-generation vascular interventional robots.

show abstract

Section: Key Technologies and Application Areas Of Vascular Intervent...mentioning

confidence: 99%

Technical and Clinical Progress on Robot-Assisted Endovascular Interventions: A Review

Duan

Akinyemi

et al. 2023

Micromachines

View full text Add to dashboard Cite

show abstract

“…As shown in Figure 1, 462 studies met the search criteria, and 21 full-text studies were assessed against the eligibility criteria. A total of 14 were identified for review (Rafii-Tari et al, 2013Chi et al, 2018aChi et al, ,b, 2020Behr et al, 2019;You et al, 2019;Zhao et al, 2019;Kweon et al, 2021;Meng et al, 2021Meng et al, , 2022Cho et al, 2022;Karstensen et al, 2022;Wang et al, 2022). The characteristics of the fourteen studies are listed in Table 2.…”

Section: Studiesmentioning

confidence: 99%

“…. RL methods RL was used in nine studies (9/14, 64%) with algorithms including A3C, DDPG, DQN, Dueling DQN, HER, PI 2 , PPO, and Rainbow (Chi et al, 2018a(Chi et al, , 2020Behr et al, 2019;You et al, 2019;Kweon et al, 2021;Meng et al, 2021Meng et al, , 2022Cho et al, 2022;Karstensen et al, 2022). Demonstrator data in some form (GAIL, Behavior Cloning, or HD) was used as a precursor in four of the studies (4/14, 29%) during training (LfD), in conjunction with other RL algorithms (Chi et al, 2018a;Behr et al, 2019;Kweon et al, 2021;Cho et al, 2022).…”

Section: Yolo Supervised Learningmentioning

confidence: 99%

“…RL methods RL was used in nine studies (9/14, 64%) with algorithms including A3C, DDPG, DQN, Dueling DQN, HER, PI 2 , PPO, and Rainbow (Chi et al, 2018a(Chi et al, , 2020Behr et al, 2019;You et al, 2019;Kweon et al, 2021;Meng et al, 2021Meng et al, , 2022Cho et al, 2022;Karstensen et al, 2022). Demonstrator data in some form (GAIL, Behavior Cloning, or HD) was used as a precursor in four of the studies (4/14, 29%) during training (LfD), in conjunction with other RL algorithms (Chi et al, 2018a;Behr et al, 2019;Kweon et al, 2021;Cho et al, 2022). The SOFA framework (Inria, Strasbourg, France; Faure et al, 2012) was used for training RL models in four studies (4/14, 29%; Behr et al, 2019;Cho et al, 2022;Karstensen et al, 2022;Meng et al, 2022), the Unity engine (Unity Technologies, San Francisco, USA) was used in two studies (2/14, 14%; You et al, 2019;Meng et al, 2021), while the platform used for training was not specified in three…”

Section: Yolo Supervised Learningmentioning

confidence: 99%

See 1 more Smart Citation

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

Robertshaw

Karstensen

Jackson

et al. 2023

Front. Hum. Neurosci.

View full text Add to dashboard Cite

BackgroundAutonomous navigation of catheters and guidewires in endovascular interventional surgery can decrease operation times, improve decision-making during surgery, and reduce operator radiation exposure while increasing access to treatment.ObjectiveTo determine from recent literature, through a systematic review, the impact, challenges, and opportunities artificial intelligence (AI) has for the autonomous navigation of catheters and guidewires for endovascular interventions.MethodsPubMed and IEEEXplore databases were searched to identify reports of AI applied to autonomous navigation methods in endovascular interventional surgery. Eligibility criteria included studies investigating the use of AI in enabling the autonomous navigation of catheters/guidewires in endovascular interventions. Following Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA), articles were assessed using Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). PROSPERO: CRD42023392259.ResultsFour hundred and sixty-two studies fulfilled the search criteria, of which 14 studies were included for analysis. Reinforcement learning (RL) (9/14, 64%) and learning from expert demonstration (7/14, 50%) were used as data-driven models for autonomous navigation. These studies evaluated models on physical phantoms (10/14, 71%) and in-silico (4/14, 29%) models. Experiments within or around the blood vessels of the heart were reported by the majority of studies (10/14, 71%), while non-anatomical vessel platforms “idealized” for simple navigation were used in three studies (3/14, 21%), and the porcine liver venous system in one study. We observed that risk of bias and poor generalizability were present across studies. No procedures were performed on patients in any of the studies reviewed. Moreover, all studies were limited due to the lack of patient selection criteria, reference standards, and reproducibility, which resulted in a low level of evidence for clinical translation.ConclusionDespite the potential benefits of AI applied to autonomous navigation of endovascular interventions, the field is in an experimental proof-of-concept stage, with a technology readiness level of 3. We highlight that reference standards with well-identified performance metrics are crucial to allow for comparisons of data-driven algorithms proposed in the years to come.Systematic review registrationidentifier: CRD42023392259.

show abstract

“…Cho et al [43] created an image-based automatic control approach for controlling the guidewire. Kweon et al [44] applied deep RL for guidewire navigation in a coronary artery phantom. For motion control during robotic catheterisation, Omisore et al [22] used a sample-efficient deep RL with episodic policy transfer and a fully adaptive tuning technique.…”

Section: Introductionmentioning

confidence: 99%

CathSim: An Open-source Simulator for Autonomous Cannulation

Jianu¹,

Huang²,

Abdelaziz³

et al. 2022

Preprint

View full text Add to dashboard Cite

Autonomous robots in endovascular operations have the potential to navigate circulatory systems safely and reliably while decreasing the susceptibility to human errors. However, there are numerous challenges involved with the process of training such robots such as long training duration due to sample inefficiency of machine learning algorithms and safety issues arising from the interaction between the catheter and the endovascular phantom. Physics simulators have been used in the context of endovascular procedures, but they are typically employed for staff training and generally do not conform to the autonomous cannulation goal. Furthermore, most current simulators are closed-source which hinders the collaborative development of safe and reliable autonomous systems. In this work, we introduce CathSim, an open-source simulation environment that accelerates the development of machine learning algorithms for autonomous endovascular navigation. We first simulate the high-fidelity catheter and aorta with the state-of-the-art endovascular robot. We then provide the capability of real-time force sensing between the catheter and the aorta in the simulation environment. We validate our simulator by conducting two different catheterisation tasks within two primary arteries using two popular reinforcement learning algorithms, Proximal Policy Optimization (PPO) and Soft Actor-Critic (SAC). The experimental results show that using our open-source simulator, we can successfully train the reinforcement learning agents to perform different autonomous cannulation tasks. Our CathSim simulator is publicly available at https://github.com/tudorjnu/cathsim.

show abstract

Deep Reinforcement Learning for Guidewire Navigation in Coronary Artery Phantom

Cited by 28 publications

References 30 publications

Technical and Clinical Progress on Robot-Assisted Endovascular Interventions: A Review

Technical and Clinical Progress on Robot-Assisted Endovascular Interventions: A Review

Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review

CathSim: An Open-source Simulator for Autonomous Cannulation

Contact Info

Product

Resources

About