Real-time hand tracking on depth images

Abstract

Hand tracking is a fundamental task in a gesture recognition system. Most previous works tracked the hand position on color images and relied heavily on skin color information. However, color information is very vulnerable to lighting variations and skin color varies across difference human races. Furthermore, one can not effectively discriminate faces or other skin-color-like objects from hands when using skin color detection. In this paper, we propose a hand tracking algorithm that uses depth images only, and also a hand click detection method to initialize the hand tracking automatically. We show that depth images suffice and are advantageous to real-time hand tracking. A region growing technique is applied to segment the hand region on depth images. Then a mean-shift based algorithm accurately locates the hand center in the segmented hand region. The experimental results show that the proposed tracking algorithm runs at 300+ FPS, and the average error of the tracked 3D hand positions is less than 1 centimeter. The proposed method enables a plethora of potential applications to natural Human-Computer Interaction (HCI), and is adequate for embedded systems of consumer electronics because of its low complexity and low bandwidth requirement.

Multi-users online recognition of technical gestures for natural human–robot collaboration in manufacturing

“…However, detection of hands in depthvideos is still challenging because of the usually rather low resolution of these sensors: only hands which are close enough to the camera can be easily segmented. Chen et al (2011) track the hands' location and segment them using a region-growing algorithm. Hamester et al (2013) detect hands in depth images based on Fourier descriptors of contours classified using a SVM.…”

Hand detection in American Sign Language depth data using domain-driven random forest regression

“…Suarez and Murphy, in their survey about gesture recognition using depth images, listed 13 methods developed by researchers for hand localization [4]. Common methods used for hand detection in depth images are depth thresholding [7], [19] and region growing [20], [21]. However, both these methods are unsuitable for ASLR applications.…”

“…However, both these methods are unsuitable for ASLR applications. Other techniques to perform hand tracking from depth information include algorithms based on Kalman filters, mean shift and continuous adaptive mean shift [20], [22], [23], [24], [25]. Shotton et al [8] have implemented a real-time human pose recognizer.…”

Multi-touchless: Real-time fingertip detection and tracking using geodesic maxima

“…This includes systems developed to track hands or fingertip locations as hand tracking is often the first stage in finding the fingertips. Using a region-growing technique Chen et al [4] locates and tracks the center of the hand. Nanda and Fujimura [11] uses Potential Fields to track a model contour over depth sequences that can be applied to hands.…”