A Human Gesture Mapping Method to Control a Multi‐Functional Hand for Robot‐Assisted Laparoscopic Surgery: The MUSHA Case

Abstract

This work presents a novel technique to control multi-functional hand for robot-assisted laparoscopic surgery. We tested the technique using the MUSHA multi-functional hand, a robot-aided minimally invasive surgery tool with more degrees of freedom than the standard commercial end-effector of the da Vinci robot. Extra degrees of freedom require the development of a proper control strategy to guarantee high performance and avoid an increasing complexity of control consoles. However, developing reliable control algorithms while reducing the control side’s mechanical complexity is still an open challenge. In the proposed solution, we present a control strategy that projects the human hand motions into the robot actuation space. The human hand motions are tracked by a LeapMotion camera and mapped into the actuation space of the virtualized end-effector. The effectiveness of the proposed method was evaluated in a twofold manner. Firstly, we verified the Lyapunov stability of the algorithm, then an user study with 10 subjects assessed the intuitiveness and usability of the system.

Simulative Evaluation of a Joint-Cartesian Hybrid Motion Mapping for Robot Hands Based on Spatial In-Hand Information

“…In an interesting more recent mapping approach denoted as DexPilot ( Handa et al, 2020 ), PH fingertip poses are obtained via a marker-less vision, and, thereafter, a cost function based on the distance and orientation among fingertips in the TH workspace is designed in order to optimize the final TH finger configurations. Other types of mapping are based on recognizing the PH posture in order to reproduce it on the TH ( Ekvall and Kragic, 2004 ; Pedro et al, 2012 ; Ficuciello et al, 2021 ). In this kind of approach, the functions of the TH are limited to a discrete set of predefined grasps/motions and, thereafter, based on the output of the PH posture recognition process (mostly based on machine learning techniques ( Dillmann et al, 2000 ; Yoshimura and Ozawa, 2012 )), one of the available TH motions is selected and executed.…”