American sign language recognition with the kinect

Zafrulla, Zahoor; Brashear, Helene; Starner﻿﻿, Thad; Hamilton, Harley; Presti, Peter

doi:10.1145/2070481.2070532

Cited by 303 publications

(124 citation statements)

References 24 publications

Supporting

Mentioning

112

Contrasting

Unclassified

Order By: Relevance

“…It is for this reason that many sign recognition approaches use variants of HMMs (Starner and Pentland, 1997;Vogler and Metaxas, 1999;Kadir et al, 2004;Cooper and Bowden, 2007). One of the first videos to be uploaded to the web came from Zafrulla et al (2011) and was an extension of their previous CopyCat game for deaf children (Zafrulla et al, 2010). The original system uses coloured gloves and accelerometers to track the hands.…”

Section: Introductionmentioning

confidence: 99%

Sign Language Recognition Using Sub-units

Cooper

Ong

Pugeault

et al. 2017

The Springer Series on Challenges in Machine Learning

View full text Add to dashboard Cite

This paper discusses sign language recognition using linguistic sub-units. It presents three types of sub-units for consideration; those learnt from appearance data as well as those inferred from both 2D or 3D tracking data. These sub-units are then combined using a sign level classifier; here, two options are presented. The first uses Markov Models to encode the temporal changes between sub-units. The second makes use of Sequential Pattern Boosting to apply discriminative feature selection at the same time as encoding temporal information. This approach is more robust to noise and performs well in signer independent tests, improving results from the 54% achieved by the Markov Chains to 76%.

show abstract

Section: Introductionmentioning

confidence: 99%

Sign Language Recognition Using Sub-units

Cooper

Ong

Pugeault

et al. 2017

The Springer Series on Challenges in Machine Learning

View full text Add to dashboard Cite

show abstract

“…With the public windows SDK, the joint locations can be obtained in real-time. For the convenience to get abundant data, Kinect has been used in gesture recognition [17], 3D body scanning [18] and sign language recognition [19,20]. Among all 20 joints provided, 5 joints including head, both hands, and both elbows are used in our implementation.…”

Section: Datasets and Experimental Settingmentioning

confidence: 99%

Curve Matching from the View of Manifold for Sign Language Recognition

Lin

Chai

Zhou

et al. 2015

Computer Vision - ACCV 2014 Workshops

View full text Add to dashboard Cite

Abstract. Sign language recognition is a challenging task due to the complex action variations and the large vocabulary set. Generally, sign language conveys meaning through multichannel information like trajectory, hand posture and facial expression simultaneously. Obviously, trajectories of sign words play an important role for sign language recognition. Although the multichannel features are helpful for sign representation, this paper only focuses on the trajectory aspect. A method of curve matching based on manifold analysis is proposed to recognize isolated sign language word with 3D trajectory captured by Kinect. From the view of manifold, the main structure of the curve is found by the intrinsic linear segments, which are characterized by some geometric features.Then the matching between curves is transformed into the matching between two sets of sequential linear segments. The performance of the proposed curve matching strategy is evaluated on two different sign language datasets. Our method achieves a top-1 recognition rate of 78.3% and 61.4% in a 370 daily words dataset and a large dataset containing 1000 vocabularies.

show abstract

“…Second, Microsoft provided a Software Development Kit for the Kinect, in which the kinematic information on the major joints of the human skeleton is available to users, thus easing the difficulty of the development work. These applications include human detection [14], physical rehabilitation [15], sign language recognition [16], etc.…”

Section: A Overview Of Common Applicationsmentioning

confidence: 99%

Kinect-based Universal Range Sensor and its Application in Educational Laboratories

Zhang

Chang

et al. 2015

Int. J. Onl. Eng.

View full text Add to dashboard Cite

Abstract-Traditional data acquisition (DAQ) systems for obtaining 3-D range information consist of sensors, DAQ measurement hardware and a processor with software. This kind of DAQ system is adapted and calibrated for various applications, thus imposing significant up-front costs that hamper its broad usage in educational laboratories. The low-cost Microsoft Kinect has become a promising alternative solution that has already been widely adopted by consumers and offers a wide variety of opportunities for being used in many other areas. The Kinect's cameras are capable of producing high quality synchronized video that consists of both color and depth data. This enables the Kinect to compete with other sophisticated 3-D sensor DAQ systems in terms of performance criteria such as accuracy, stability, reliability and error rates. One of the most noticeable Kinect-based applications is the skeleton-based tracking of humans, which is made possible by its built-in human skeleton recognition functions. Other common usages of the Kinect are 3-D surface/scene reconstruction and object classification/recognition. However, Kinect-based developments that focus on the tracking of arbitrary objects have rarely been reported, mainly due to a lack of mature algorithms.In the first part of this paper, a three-stage approach for capturing general motions of objects will be introduced. This approach consists of point cloud pre-processing with a focus on computational efficiency, object tracking employing recognition and post-processing including motion analysis. This approach can be tailored to special cases, namely the algorithms focus more on computation efficiency when the objects of interest have simple shapes or colors, and they focus more on reliability for objects with complex geometries or textures. The second part of this paper describes the integration of the proposed DAQ system into a multi-player game-based laboratory environment. In this implementation, a physical experiment is triggered by the game avatars and the experimental data acquired by the Kinect and analyzed by the proposed algorithms are then fed back into the game environment and used to animate the experimental device.

show abstract

American sign language recognition with the kinect

Cited by 303 publications

References 24 publications

Sign Language Recognition Using Sub-units

Sign Language Recognition Using Sub-units

Curve Matching from the View of Manifold for Sign Language Recognition

Kinect-based Universal Range Sensor and its Application in Educational Laboratories

Contact Info

Product

Resources

About