Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera–A Pilot Study

Abstract

We present a gesture recognition system for recognizing hand movements in near realtime. The system uses a infra-red time-of-flight range camera with up to 30 Hz framerate to measure 3d-surface points captured from the hand of the user. The measured data is transformed into a cloud of 3d-points after depth keying and suppression of camera noise by median filtering. Principle component analysis (PCA) is used to obtain a first crude estimate on the location and orientation of the hand. An articulated hand model is fitted to the data to refine the first estimate. The unoptimized system is able to estimate the first 7 Degrees of Freedom of the hand within 200 ms. The reconstructed hand is visualized in AVANGO/Performer and can be utilized to implement a natural man-machine interface. The work reviews relevant publications, underlines the advantages and shortcomings of the approach and provides an outlook on future improvements.

Real-time Motion-based Hand Gestures Recognition from Time-of-Flight Video

“…In [22] some advantages are remarked: robustness to illumination changes and easy segmentation even when there is camera motion. In [2] a 3D hand model is adjusted to the cloud of points obtained from the captured depth image. In [17,31,25,27,26,29] experiments, using depth sensors, are performed over static hand gestures collections, pointing out the advantages of using depth information.…”

Fingertip-based Real Time Tracking and Gesture Recognition for Natural User Interfaces

“…Some of the cutting-edge technologies employed by these solutions are briefly presented in this research, of those, the most promising technologies are the 3D acquisition solutions based on either structured light approach [2][3] or the Timeof-Flight (TOF) principle [4][5][6]. One of the most important advantage of these approaches is simplifying the hand segmentation.…”

3D hand tracking using Kalman filter in depth space

“…After the emergence of this equipment, real-time 3D hand tracking methods rapidly developed. For example, Breuer et al [12] used an infra-red ToF camera to create a near real-time gesture recognition system. Grest et al [13] proposed a human motion tracking method using a combination of depth and silhouette information.…”