Deep Reinforcement Learning for the Navigation of Neurovascular Catheters

Behr, Tobias; Pusch, Tim Philipp; Siegfarth, Marius; Hüsener, Dominik; Mörschel, Tobias; Karstensen, Lennart

doi:10.1515/cdbme-2019-0002

Cited by 26 publications

(39 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper extends own previous work, where initial navigation trials of an autonomous catheter guidance through a simple transparent acrylic glass vascular phantom is shown [6]. The simulation framework, testbench manipulator and camera-based guidewire tracking are reused.…”

Section: Introductionmentioning

confidence: 67%

Autonomous guidewire navigation in a two dimensional vascular phantom

Karstensen

Behr

Pusch

et al. 2020

Current Directions in Biomedical Engineering

Self Cite

View full text Add to dashboard Cite

The treatment of cerebro- and cardiovascular diseases requires complex and challenging navigation of a catheter. Previous attempts to automate catheter navigation lack the ability to be generalizable. Methods of Deep Reinforcement Learning show promising results and may be the key to automate catheter navigation through the tortuous vascular tree. This work investigates Deep Reinforcement Learning for guidewire manipulation in a complex and rigid vascular model in 2D. The neural network trained by Deep Deterministic Policy Gradients with Hindsight Experience Replay performs well on the low-level control task, however the high-level control of the path planning must be improved further.

show abstract

Section: Introductionmentioning

confidence: 67%

Autonomous guidewire navigation in a two dimensional vascular phantom

Karstensen

Behr

Pusch

et al. 2020

Current Directions in Biomedical Engineering

Self Cite

View full text Add to dashboard Cite

show abstract

“…Considering that simple control operations are repeatedly performed to utilize a robot-assisted intervention system, deep RL may be a solution that effectively alleviates the burden of human operators. Recent applications for autonomous control of interventional devices in phantom simulation supported the potential applicability of deep RL (You et al, 2019;Behr et al, 2019;Karstensen et al, 2020;Chi et al, 2020;Zhao et al, 2019). In this study, we propose a deep RL framework for autonomous guidewire navigation in robot-assisted coronary interventions.…”

mentioning

confidence: 88%

Deep reinforcement learning for guidewire navigation in coronary artery phantom

Kweon,

Kim,

Lee

et al. 2021

Preprint

View full text Add to dashboard Cite

In percutaneous intervention for treatment of coronary plaques, guidewire navigation is a primary procedure for stent delivery. Steering a flexible guidewire within coronary arteries requires considerable training, and the non-linearity between the control operation and the movement of the guidewire makes precise manipulation difficult. Here, we introduce a deep reinforcement learning (RL) framework for autonomous guidewire navigation in a robot-assisted coronary intervention. Using Rainbow, a segment-wise learning approach is applied to determine how best to accelerate training using human demonstrations with deep Q-learning from demonstrations (DQfD), transfer learning, and weight initialization. 'State' for RL is customized as a focus window near the guidewire tip, and subgoals are placed to mitigate a sparse reward problem. The RL agent improves performance, eventually enabling the guidewire to reach all valid targets in 'stable' phase. Our framework opens a new direction in the automation of robot-assisted intervention, providing guidance on RL in physical spaces involving mechanical fatigue.

show abstract

“…As soft robots can be tricky to model, a tempting solution is to adopt a "model-free" approach and learn the control policy directly [51]. Several Reinforcement Learning algorithms have been used for continuum robot control (such as catheter navigation in medical robotics [52]) and soft robotics control. Applications include manipulation and navigation tasks.…”

Section: Learning Based Controlmentioning

confidence: 99%

Review on generic methods for mechanical modeling, simulation and control of soft robots

Schegg

Duriez

2022

PLoS ONE

View full text Add to dashboard Cite

In this review paper, we are interested in the models and algorithms that allow generic simulation and control of a soft robot. First, we start with a quick overview of modeling approaches for soft robots and available methods for calculating the mechanical compliance, and in particular numerical methods, like real-time Finite Element Method (FEM). We also show how these models can be updated based on sensor data. Then, we are interested in the problem of inverse kinematics, under constraints, with generic solutions without assumption on the robot shape, the type, the placement or the redundancy of the actuators, the material behavior… We are also interested by the use of these models and algorithms in case of contact with the environment. Moreover, we refer to dynamic control algorithms based on mechanical models, allowing for robust control of the positioning of the robot. For each of these aspects, this paper gives a quick overview of the existing methods and a focus on the use of FEM. Finally, we discuss the implementation and our contribution in the field for an open soft robotics research.

show abstract

Deep Reinforcement Learning for the Navigation of Neurovascular Catheters

Cited by 26 publications

References 5 publications

Autonomous guidewire navigation in a two dimensional vascular phantom

Autonomous guidewire navigation in a two dimensional vascular phantom

Deep reinforcement learning for guidewire navigation in coronary artery phantom

Review on generic methods for mechanical modeling, simulation and control of soft robots

Contact Info

Product

Resources

About