Implementation and evaluation of a gesture-based input method in robotic surgery

Abstract

Abstract-The introduction of robotic master-slave systems for minimally invasive surgery has created new opportunities in assisting surgeons with partial or fully autonomous functions. While autonomy is an ongoing field of research, the question of how the growing number of offered features can be triggered in a time-saving manner at the master console is not well investigated. We have implemented a gesturebased user interface, whereas the haptic input devices that are commonly used to control the surgical instruments, are used to trigger actions. Intuitive and customizable gestures are learned by the system once, linked to a certain command, and recalled during operation as the gesture is presented by the surgeon. Experimental user studies with 24 participants have been conducted to evaluate the efficiency, accuracy and user experience of this input method compared to a traditional menu. The results have shown the potential of gesture-based input, especially in terms of time savings and enhanced user experience.

Overview of some Command Modes for Human-Robot Interaction Systems

“…This can happen, for instance, in noisy places, in underwater areas, and in empty space where the medium cannot convoy the voice waves as with astronauts (Liu et al, 2016). It can be also used for learning by demonstration and for imitation as reproducing operations in medical care or tele-surgery (Staub et al, 2011). This command mode is also interesting in some military activity to communicate and direct remote teams, autonomous and unmanned systems (Elliott et al, 2016).…”

Image Processing Methods for Gesture-Based Robot Control

“…This can happen in noisy places, in underwater areas, and in empty space where the medium cannot convey the sound waves as with astronauts [6]. It can be also used for learning by demonstration and for imitation as reproducing operations in medical care or tele-surgery [7]. This interaction mode is also interesting in some military activity to communicate and direct remote teams, autonomous and unmanned systems [8].…”

Safe Testing of Electrical Diathermy Cutting Using a New Generation Soft Manipulator

“…However, this implies significant material deformations [9] and viscoelastic effects [5] that make modeling the kinematics [10]- [13] and dynamics [14]- [16] of soft continuum robots challenging. Said complexity affects real-time control of such a surgical robot, which constitutes one of the main open research fronts in this field together with reliable direct contact force sensing [6] and feedback for palpation [17] and grip control [18], imaging and localization of the surgical robot within the patient [19]- [21], and tele-operation of the robot [22]- [24].…”