Context-based hand gesture recognition for the operating room

Jacob, Mithun George; Wachs, Juan P.

doi:10.1016/j.patrec.2013.05.024

Cited by 68 publications

(33 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Each HMM comprised five states in a left-to-right configuration and was trained using the Baum-Welch algorithm, which has been shown to generate promising results in hand gesture recognition (Jacob and Wachs, 2014). An observation was classified based on the specific HMM chain that best explained that observation, that is, by determining which of the trained HMM outputs had the highest probability for a state sequence⃗ z given a new observation g u and its intrinsic parameters (the initial state distribution vector π, the transition matrix A, and the observation probability within each state B), and thereby assigning the corresponding label to the new sample.…”

Section: Classification Algorithmsmentioning

confidence: 99%

“…Researchers have been studying how gestures are produced, perceived, and mimicked, as well as how computer systems can detect and recognize them. This last area is especially relevant to human-computer interaction (Pavlovic et al, 1997;Rautaray and Agrawal, 2015), HRI (Nickel and Stiefelhagen, 2007;Yang et al, 2007), and assistive technologies (Jacob and Wachs, 2014;Jiang et al, 2016b), where humans rely on accurate recognition by machines.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Human-Centered Approach to One-Shot Gesture Learning

Cabrera

Wachs

2017

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

This article discusses the problem of one-shot gesture recognition using a humancentered approach and its potential application to fields such as human-robot interaction where the user's intentions are indicated through spontaneous gesturing (one shot). Casual users have limited time to learn the gestures interface, which makes one-shot recognition an attractive alternative to interface customization. In the aim of natural interaction with machines, a framework must be developed to include the ability of humans to understand gestures from a single observation. Previous approaches to oneshot gesture recognition have relied heavily on statistical and data-mining-based solutions and have ignored the mechanisms that are used by humans to perceive and execute gestures and that can provide valuable context information. This omission has led to suboptimal solutions. The focus of this study is on the process that leads to the realization of a gesture, rather than on the gesture itself. In this case, context involves the way in which humans produce gestures-the kinematic and anthropometric characteristics. In the method presented here, the strategy is to generate a data set of realistic samples based on features extracted from a single gesture sample. These features, called the "gist of a gesture," are considered to represent what humans remember when seeing a gesture and, later, the cognitive process involved when trying to replicate it. By adding meaningful variability to these features, a large training data set is created while preserving the fundamental structure of the original gesture. The availability of a large data set of realistic samples allows the use of training classifiers for future recognition. The performance of the method is evaluated using different lexicons, and its efficiency is compared with that of traditional N-shot learning approaches. The strength of the approach is further illustrated through human and machine recognition of gestures performed by a dual-arm robotic platform.

show abstract

Section: Classification Algorithmsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Human-Centered Approach to One-Shot Gesture Learning

Cabrera

Wachs

2017

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

show abstract

“…Los primeros usos de este dispositivo para tareas ajenas a su concepción inicial de instrumento para video juegos, se dieron en sistemas de reconocimiento de gestos de manera general ( [15], [16], [17], [18]). Posteriormente se empezó a utilizar el dispositivo en diversas aplicaciones, entre ellas la replicación de movimientos en robots ( [19], [20]), la rehabilitación de pacientes [21], [22] y, finalmente, su uso como interfaz natural para el control de dispositivos quirúrgicos, como el caso del uso del Kinect para reconocer un protocolo de gestos en la inserción de guías dentro del paciente [23]; la evaluación del mismo dispositivo como generador de movimientos en un simulador quirúrgico [24], o el uso del Kinect para el reconocimiento de un protocolo de gestos en una biopsia cerebral [25]. El mayor aporte de este proyecto es la utilización del Kinect para mover dos robots quirúrgicos en operaciones de laparoscopia.…”

Section: Dispositivo Kinect Y Su Interacción Con Robosurgeryunclassified

Sistema de captura de gestos con KINECT para la manipulación de robots quirúrgicos virtuales

Hurtado-Chaves¹,

Nástar-Guacales²,

Vivas-Albán³

2015

Iteckne

View full text Add to dashboard Cite

Resumen-Este artículo presenta la aplicación de un sistema de captura de gestos con el fin de manipular dos robots quirúr-gicos virtuales. El sistema de captura de gestos se realiza por medio de un dispositivo Kinect, el cual detecta el movimiento de las manos del usuario con el fin de mover los robots quirúr-gicos, y de su rodilla derecha, utilizada para cambiar los instrumentos quirúrgicos de cada robot. Este sistema de captura con interfaz natural se prueba en un simulador quirúrgico para operaciones de laparoscopia, el cual consta de un robot porta endoscopio que se manipula con un joystick, y los dos robots quirúrgicos cuyos órganos terminales son guiados a partir de las señales captadas por el dispositivo Kinect, las cuales son transformadas en movimientos articulares de los robots debido al modelo cinemático. El sistema se prueba en una rutina de extracción de vesícula (colecistectomía) robotizada, permitiendo al usuario un mejor manejo gracias al uso de esta interfaz natural.Palabras clave-Dispositivo Kinect, interfaces naturales, robots quirúrgicos, simuladores quirúrgicos.Abstract-This article presents a gesture capture system for manipulating two virtual surgical robots. Gesture capture uses Kinect device to detect the movement of the user´s hand to move the surgical robots, and his right knee to change the needed surgical tools. This natural interface capture system is proved on a surgical simulator for laparoscopic, that has a holder endoscopic robot manipulated by a joystick, and two surgical robots that are guided by the Kinect device. Kinematics model is used to transform Kinect captured signals to articular movements for the robots. System is tested on a robotic cholecystectomy, allowing a more natural manipulation to the user.

show abstract

“…There are very interesting published over the past two decades. The close problem statements are contained in [1,2,3,4,5]. In these papers a way to assess the quality of the medicine sterile interfaces is given.…”

Section: Overviewmentioning

confidence: 99%