Kinect Sensor-Based Long-Distance Hand Gesture Recognition and Fingertip Detection with Depth Information

Abstract

Gesture recognition is an important part of human-robot interaction. In order to achieve fast and stable gesture recognition in real time without distance restrictions, this paper presents an improved threshold segmentation method. The improved method combines the depth information and color information of a target scene with hand position by the spatial hierarchical scanning method; the ROI in the scene is thus extracted by the local neighbor method. In this way, the hand can be identified quickly and accurately in complex scenes and different distances. Furthermore, the convex hull detection algorithm is used to identify the positioning of fingertips in ROI, so that the fingertips can be identified and located accurately. The experimental results show that the hand position can be obtained quickly and accurately in the complex background by using the improved method, the real-time recognition distance interval can be reached by 0.5 m to 2.0 m, and the fingertip detection rates can be reached 98.5% in average. Moreover, the gesture recognition rates are more than 96% by the convex hull detection algorithm. It can be thus concluded that the proposed method achieves good performance of hand detection and positioning at different distances.

A Novel Human-3DTV Interaction System Based on Free Hand Gestures and a Touch-Based Virtual Interface

“…Most conventional gesture recognition algorithms [10], [11] are based on RGB color images. Recently, with the development of inexpensive depth cameras like Microsoft Kinect [12], the availability of capturing the depth information along with the color information improves the gesture recognition performance [13], [14] and makes gesture recognition based on Kinect easier and popular in Human-computer interaction. Existing free-hand gesture recognition approaches, however, still suffers from several issues.…”

Hand Gesture Recognition Based on Single-Shot Multibox Detector Deep Learning

“…Due to its natural human-computer interaction characteristics, the hand gesture recognition system allows users to interact intuitively and effectively through a computer interface [5,6]. Additionally, gesture recognition based on vision is widely applied in artificial intelligence, virtual reality, multimedia, and natural language communication [7][8][9][10].…”

Dynamic Hand Gesture to Text using Leap Motion

“…Other examples of data gloves are such as the Sayre Glove, CyberGloves, and Polhemus. Nonetheless, the use of data gloves is costly and inconvenience for users to wear, as they are typically wired and connected to computers [10]. In contrast, vision-based technology is more prevalent in hand gesture recognition with lower cost, noncontact, natural and intuitive [10,11,17].…”

“…Nonetheless, the use of data gloves is costly and inconvenience for users to wear, as they are typically wired and connected to computers [10]. In contrast, vision-based technology is more prevalent in hand gesture recognition with lower cost, noncontact, natural and intuitive [10,11,17]. Vision-based approaches record hand gestures by using a camera, and the captured image or video is processed and analysed with computer vision techniques.…”

“…Microsoft's Kinect is a depth camera that acquires gestures with rich depth information and has demonstrated a notable performance of light sensitivity in gesture recognition [18]. However, there are still unresolved issues regarding gesture finding and segmenting with Kinect sensors [10]. Different from the Kinect, the Leap Motion controller is a device explicitly targeted to hand gesture recognition and can directly compute the position of the fingertips and the hand orientation with better accuracy [13].…”