Collaborative Suturing: A Reinforcement Learning Approach to Automate Hand-off Task in Suturing for Surgical Robots

Varier, Vignesh Manoj; Rajamani, Dhruv Kool; Goldfarb, Nathaniel; Tavakkolmoghaddam, Farid; Munawar, Adnan; Fischer, Gregory S.

doi:10.1109/ro-man47096.2020.9223543

Cited by 37 publications

(14 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the suture task, we use one more angular movement (4 dimensions), resulting in 81 possible discrete actions. A smaller action space naturally results in faster convergence rates, since the resulting state-action space is smaller [24].…”

Section: B State and Action Spacementioning

confidence: 99%

Robotic Surgery With Lean Reinforcement Learning

Barnoy,

O'Brien,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

As surgical robots become more common, automating away some of the burden of complex direct human operation becomes ever more feasible. Model-free reinforcement learning (RL) is a promising direction toward generalizable automated surgical performance, but progress has been slowed by the lack of efficient and realistic learning environments. In this paper, we describe adding reinforcement learning support to the da Vinci Skill Simulator, a training simulation used around the world to allow surgeons to learn and rehearse technical skills. We successfully teach an RL-based agent to perform sub-tasks in the simulator environment, using either image or state data. As far as we know, this is the first time an RL-based agent is taught from visual data in a surgical robotics environment. Additionally, we tackle the sample inefficiency of RL using a simple-to-implement system which we term hybrid-batch learning (HBL), effectively adding a second, long-term replay buffer to the Q-learning process. Additionally, this allows us to bootstrap learning from images from the data collected using the easier task of learning from state. We show that HBL decreases our learning times significantly.

show abstract

Section: B State and Action Spacementioning

confidence: 99%

Robotic Surgery With Lean Reinforcement Learning

Barnoy,

O'Brien,

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Q-learning is a form of model-free RL, and may be the most promising form of RL for autonomous actions in surgery as it most closely resembles the human experience [ 84 ]. A Q-learning form of RL was also recently developed using the da Vinci Research Kit (Intuitive Surgical, Sunnyvale, CA, USA) to create a system to perform autonomous surgical knots using the robot [ 85 ]. However, as in current surgery, what we gain in speed we often end up paying for in increased complications, which brings up the important question of ethics in AI surgical innovation.…”

Section: Deep Learning and Computer Visionmentioning

confidence: 99%

Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?

Gumbs

Frigerio

Spolverato

et al. 2021

Sensors

View full text Add to dashboard Cite

Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner.

show abstract

“…Six common surgical tasks appear to be widely investigated for automation. For the suturing task, including the works related to knot tying and needle insertion, we reported the following: [16], [70], [85], [88], [89], [161], [178], [195]- [197], [203], [243], and [246]. The pick, transfer, and place task was mainly characterized by experiments relying on pegs and rings from the fundamentals of the laparoscopic surgery training paradigm [54], [69], [91], [96], [102], [151]- [153], [163], [264] or new surgical tools [177].…”

Section: Instrument Controlmentioning

confidence: 99%

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

D'Ettorre

Mariani

Stilli

et al. 2021

IEEE Robot. Automat. Mag.

View full text Add to dashboard Cite

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.R obotic-assisted surgery is now well established in clinical practice and has become the gold-standard clinical treatment option for several clinical indications. The field of robotic-assisted surgery is expected to grow substantially in the next decade, with a range of new robotic devices emerging to address unmet clinical needs across different specialties. A vibrant surgical robotics research community is pivotal for conceptualizing such new systems as well as for developing and training the engineers and scientists to translate them into practice. The da Vinci Research Kit (dVRK), an academic and industry collaborative effort to repurpose decommissioned da Vinci surgical systems [Intuitive Surgical Inc. (ISI), California, USA] as a research platform for surgical robotics research, has been a key initiative for addressing a barrier to entry for new research groups in surgical robotics. In this article, we present an extensive review of the publications that have been facilitated by the dVRK over the past decade. We classify research efforts into different categories and outline some of the major challenges and needs for the robotics community to maintain and build upon this initiative.

show abstract

Collaborative Suturing: A Reinforcement Learning Approach to Automate Hand-off Task in Suturing for Surgical Robots

Cited by 37 publications

References 18 publications

Robotic Surgery With Lean Reinforcement Learning

Robotic Surgery With Lean Reinforcement Learning

Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?

Accelerating Surgical Robotics Research: A Review of 10 Years With the da Vinci Research Kit

Contact Info

Product

Resources

About