Virtual Fixture Assistance for Suturing in Robot-Aided Pediatric Endoscopic Surgery

Marinho, Murilo M.; Ishida, Hisashi; Harada, Kensuke; Deie, Kyoichi; Mitsuishi, Mamoru

doi:10.1109/lra.2019.2963642

Cited by 30 publications

(33 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Research covering some aspects of this question exists. Marinho et al proposed rendering VFs in the joint space of the RR to incorporate the redundancy resolution into the constraints imposed by VFs (Marinho et al, 2019(Marinho et al, , 2020. Rahal et al used the so-called HI-side shared control, where the HI's position and orientation were used instead of RR's to compute the haptic-feedback commands (Rahal et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

“…Because of their simplicity and intuitiveness, VFs are promising methods that have been shown to improve operators' teleoperation performance, reduce total workload, and ensure the safety of task execution (Rosenberg 1993;Pruks et al, 2018;Moccia et al 2019Moccia et al , 2020Marinho et al, 2020;Pruks and Ryu 2020). However, most extant research considered VFs to be pre-defined or automatically generated.…”

Section: Introductionmentioning

confidence: 99%

“…For example, accurate VF models of the human eyeball are used by medical professionals to improve safety during robotic-assisted eye surgery (Ebrahimi et al, 2019; Smits et al, 2019; Yang et al, 2019). Additionally, VFs associated with RR tools help prevent collision among multiple RRs operating in proximity (Marinho et al, 2020; Moccia et al, 2020). They also help RRs perform complicated motions, such as knot-tying (Chen et al, 2016), suturing (Marinho et al, 2020), and manufacture monitoring (Sita et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

Pruks¹,

Ryu

2022

The International Journal of Robotics Research

View full text Add to dashboard Cite

A virtual fixture (VF) is a constraint built into software that assists a human operator in moving a remote tool along a preferred path via an augmented guidance force to improve teleoperation performance. However, teleoperation generally applies to unknown or dynamic environments, which are challenging for VF use. Most researchers have assumed that VFs are pre-defined or generated automatically; however, these processes are complicated and unreliable in unknown environments where teleoperation is in high demand. Recently, a few researchers have addressed this issue by introducing a user-interactive method of generating VFs in unknown environments. However, these methods are limited to generating a single type of primitive for a single robot tool. Moreover, the accuracy of the VF generated by these methods depends on the accuracy of the human input. Thus, applications of these methods are limited. To overcome those limitations, this work introduces a novel interactive VF generation method that includes a new method of representing VFs as a composition of components. A feature-based user interface allows the human operator to intuitively specify the VF components. The new VF representation accommodates a variety of robot tools and actions. Using the feature-based interface, the process of VF generation is more intuitive and accurate. In this study, the proposed method is evaluated with human subjects in three teleoperation experiments: peg-in-hole, pipe-sawing, and pipe-welding. The experimental results show that the VFs generated by the proposed approach result in a higher manipulation quality while demonstrating the lowest total workload in all experiments. The peg-in-hole task teleoperation was the safest in terms of failure proportion and exerted force of the robot tool. In the pipe-sawing task, the positioning of the robot tool was the most accurate. In the pipe-welding task, the quality of weld was the best in terms of measured tool-trajectory smoothness and visual weld observation.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

Pruks¹,

Ryu

2022

The International Journal of Robotics Research

View full text Add to dashboard Cite

show abstract

“…Selvaggio et al [29] presented an optimization-based haptic shared control for needle grasping that takes into consideration the robot joint limits and singularities. Marinho et al [30] developed a looping guidance virtual fixture and a trajectory guidance cylinder based on constrained optimization and haptic feedback to assist the surgeon during a teleoperated knot-tying task. In [31], multiple control strategies for a stitching task were compared: telemanipulation, autonomous, shared control with orientation free, and shared control with orientation constrained.…”

Section: Related Workmentioning

confidence: 99%

Optimization-Based Constrained Trajectory Generation for Robot-Assisted Stitching in Endonasal Surgery

et al. 2021

View full text Add to dashboard Cite

The reduced workspace in endonasal endoscopic surgery (EES) hinders the execution of complex surgical tasks such as suturing. Typically, surgeons need to manipulate non-dexterous long surgical instruments with an endoscopic view that makes it difficult to estimate the distances and angles required for precise suturing motion. Recently, robot-assisted surgical systems have been used in laparoscopic surgery with promising results. Although robotic systems can provide enhanced dexterity, robot-assisted suturing is still highly challenging. In this paper, we propose a robot-assisted stitching method based on an online optimization-based trajectory generation for curved needle stitching and a constrained motion planning framework to ensure safe surgical instrument motion. The needle trajectory is generated online by using a sequential convex optimization algorithm subject to stitching kinematic constraints. The constrained motion planner is designed to reduce surrounding damages to the nasal cavity by setting a remote center of motion over the nostril. A dual concurrent inverse kinematics (IK) solver is proposed to achieve convergence of the solution and optimal time execution, in which two constrained IK methods are performed simultaneously; a task-priority based IK and a nonlinear optimization-based IK. We evaluate the performance of the proposed method in a stitching experiment with our surgical robotic system in a robot-assisted mode and an autonomous mode in comparison to the use of a conventional surgical tool. Our results demonstrate a noticeable improvement in the stitching success ratio in the robot-assisted mode and the shortest completion time for the autonomous mode. In addition, the force interaction with the tissue was highly reduced when using the robotic system.

show abstract

“…Another recent medical application has been reported by a group from the University of Tokyo (Marinho et al , 2020). The limited workspace available in paediatric endoscopic surgery makes suturing a particularly difficult task and surgeons have to prevent collisions between tools and also with the surrounding tissues.…”

Section: The Role Of Virtual Fixturesmentioning

confidence: 99%

The role of augmented reality in robotics

Bogue

2020

View full text Add to dashboard Cite

Purpose The purpose of this paper is to provide an insight into how augmented reality (AR) technologies are being applied to robotics. Design/methodology/approach Following an introduction and a brief historical background to AR, this first provides examples of AR applications in robot programming. It then gives examples of recent research into AR-based robot teleoperation. Research activities involving the virtual fixtures (VF) technique are then discussed and finally, brief conclusions are drawn. Findings Because AR concepts were first investigated in the 1990s, applications involving robotics have been widely studied. Programming with the aid of AR devices, such as the HoloLens headset, can be simplified and AR methods, including the VF technique, can improve the accuracy and speed of teleoperation, manipulation and positional control tasks. They can also provide visual or haptic feedback which leads to more intuitive operation and significantly reduces the cognitive load on the operator. Originality/value This provides an insight into the growing role of AR in robotics by providing examples of recent research in a range of applications.

show abstract

Virtual Fixture Assistance for Suturing in Robot-Aided Pediatric Endoscopic Surgery

Cited by 30 publications

References 26 publications

Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

Method for generating real-time interactive virtual fixture for shared teleoperation in unknown environments

Optimization-Based Constrained Trajectory Generation for Robot-Assisted Stitching in Endonasal Surgery

The role of augmented reality in robotics

Contact Info

Product

Resources

About