A Framework for Recognizing the Simultaneous Aspects of American Sign Language

Abstract

The major challenge that faces American Sign Language (ASL) recognition now is developing methods that will scale well with increasing vocabulary size. Unlike in spoken languages, phonemes can occur simultaneously in ASL. The number of possible combinations of phonemes is approximately 1.5 × 10 9 , which cannot be tackled by conventional hidden Markov model-based methods. Gesture recognition, which is less constrained than ASL recognition, suffers from the same problem. In this paper we present a novel framework to ASL recognition that aspires to being a solution to the scalability problems. It is based on breaking down the signs into their phonemes and modeling them with parallel hidden Markov models. These model the simultaneous aspects of ASL independently. Thus, they can be trained independently, and do not require consideration of the different combinations at training time. We show in experiments with a 22-sign-vocabulary how to apply this framework in practice. We also show that parallel hidden Markov models outperform conventional hidden Markov models.

Fingerspelling Recognition through Classification of Letter-to-Letter Transitions

“…Techniques differ mainly in the degree to which handshape information is considered. Some methods [2,3] use only very basic handshape information, if any; others [4] use a complete description of the bending angles at 18 joints in the hand, which are measured using an instrumented glove such as a CyberGlove.…”

Survey on Gesture Recognition for Hand Image Postures

“…It can be used for disabled peopled when communicating with the other people, and with the computer as well (Murthy, 2009). American Sign Language in (Vogler, 2001;Starner, 2002) is one example that has received significant attention in the gesture literature. Kim (1996) recognized Korean Sign Language (KSL).…”

Smoothed Disparity Maps for Continuous American Sign Language Recognition

“…Most of the current sign language recognition systems use specialised hardware [3,16] and are person dependent [13], i.e. can only recognise the one signer it was designed for.…”