Fast-gesture recognition and classification using Kinect: an application for a virtual reality drumkit

Rosa-Pujazón, Alejandro; Barbancho, Isabel; Tardón, Lorenzo J.; Barbancho, Ana M.

doi:10.1007/s11042-015-2729-8

Cited by 11 publications

(8 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work is focused on how the human poses extracted by OpenPose can be processed and used as input signals for Dynamic Time Warping in such a way that these two components form a processing pipeline which ultimately yields a classification of human gestures using only RGB images. What sets this apart from proposed methods based on the Microsoft Kinect [23] [3][5] is that we do not use depth data and extract the pose key points ourselves instead of relying on ones provided by the Kinect framework. This makes our approach independent of any special sensor hardware.…”

Section: Related Workmentioning

confidence: 99%

“…Moreover, we want to avoid the need of any specific hardware. For example, the Microsoft Kinect is a popular platform for training models and collecting data for gesture recognition [3] [4] [5] [6], since it provides not only an RGB video but also depth data. The task of gesture recognition can be simplified by placing special markers on the person's body [7] or special gloves for hand gestures [8].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Schneider

Memmesheimer

Kramer

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Gesture recognition opens up new ways for humans to intuitively interact with machines. Especially for service robots, gestures can be a valuable addition to the means of communication to, for example, draw the robot's attention to someone or something. Extracting a gesture from video data and classifying it is a challenging task and a variety of approaches have been proposed throughout the years. This paper presents a method for gesture recognition in RGB videos using OpenPose to extract the pose of a person and Dynamic Time Warping (DTW) in conjunction with One-Nearest-Neighbor (1NN) for time-series classification. The main features of this approach are the independence of any specific hardware and high flexibility, because new gestures can be added to the classifier by adding only a few examples of it. We utilize the robustness of the Deep Learning-based OpenPose framework while avoiding the data-intensive task of training a neural network ourselves. We demonstrate the classification performance of our method using a public dataset.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Schneider

Memmesheimer

Kramer

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…In an environment where human and computers are not connected directly, voice and body movements can be used to express our ideas in a good degree [7], [12], [14]. In terms of body movements, as the most flexible part of human's body, hand can be used most frequently to express us when we interact with computers in VR environment [15], [16].…”

Section: Introductionmentioning

confidence: 99%

Interaction with Three-Dimensional Gesture and Character Input in Virtual Reality: Recognizing Gestures in Different Directions and Improving User Input

Yang

Wang

et al. 2018

IEEE Consumer Electron. Mag.

View full text Add to dashboard Cite

Hand gesture recognition is a key aspect to make interaction of virtual reality more convenient. A good way to make users' idea understood by computers including characters input plays an important role in interaction. Current methods of hand gesture and character input are too limited to make full use of powerful capacity that computers have nowadays. In this paper, we propose a natural 3D input method based on stereo cameras as an interface of human and machine. We segment the hand out based on skin-color detection and train a neural network based on Hu moments to recognize valid and invalid gestures defined in our paper. For valid gestures, we implement stereo matching and 3D coordinate calculation and line them up to formulate characters. Our method can robustly recognize 3D gestures in different directions and make users' input more free compared with traditional ways.

show abstract

“…Detection and classification of dance figures using the Kinect sensor can be performed in real time [6]. In drumkit simulator application, recognition and classification of drumkit actions can be realized in real time using kinect sensor [7].…”

Section: Introductionmentioning

confidence: 99%

“…Another solution is to use features with less size instead of using all the joint information from the position information in the human body skeleton representation. Instead of the trajectory of all body joints, action recognition can be performed using the trajectory of hand joint and the shoulder position [7]. The use of only significant joint trajectory rather than using all the joint information, can provide a significant increase in performance.…”

Section: Introductionmentioning

confidence: 99%

Human Gesture Recognition using Keyframes on Local Joint Motion Trajectories

Durgut¹,

Fındık²

2017

ijacsa

View full text Add to dashboard Cite

Abstract-Human Action Recognition (HAR) systems are systems that recognize and classify the actions that users perform against the sensor or camera. In most HAR systems, an input test data is compared with the reference data in the database using various methods. Classification process is performed according to the result obtained. The size of the test or reference data directly affects the operation speed of the system. Reduced data size allows a significant performance increase in system operation speed. In this study, action recognition method is proposed by using skeletal joint information obtained by Microsoft Kinect sensor. Splitting keyframes are obtained from the skeletal joint information. The keyframes are observed as a distinguishing feature. Therefore, these keyframes are used for the classification process. Keeping the keyframes instead of keeping the position or angle information of action in the reference database can benefit from memory and working time. The weight value of each keyframes is calculated in the method. The problem of temporal differences that occur when comparing test and reference action is solved by Dynamic Time Warping (DTW). The k-nearest neighbor's algorithm is used for classification according to the obtained results from DTW. The sample has been tested in a data set so that the success of the method can be tested. As a result, 100% correct classification was achieved. It is also suitable for working at real time systems. Breakpoints can also be used to provide feedback to the user as a result of the classification process. The magnitude and direction of the keyframes, the change in the trajectory of joint, the position and the time of its existence also give information about the time errors.

show abstract

Fast-gesture recognition and classification using Kinect: an application for a virtual reality drumkit

Cited by 11 publications

References 31 publications

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Gesture Recognition in RGB Videos Using Human Body Keypoints and Dynamic Time Warping

Interaction with Three-Dimensional Gesture and Character Input in Virtual Reality: Recognizing Gestures in Different Directions and Improving User Input

Human Gesture Recognition using Keyframes on Local Joint Motion Trajectories

Contact Info

Product

Resources

About