American Sign Language Phrase Verification in an Educational Game for Deaf Children

Zafrulla, Zahoor; Brashear, Helene; Yin, Ping; Presti, Peter; Starner﻿﻿, Thad; Hamilton, Harley

doi:10.1109/icpr.2010.937

Cited by 35 publications

(19 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Recently, Zafrulla et al (2010) have performed real-time ASL phrase verification for an educational game, Copycat using HMMs, by applying a rejection threshold on the probability of the observed sequence for each sign in the phrase. They have tested this approach using 1,204 signed phrase samples from 11 deaf children during live game play of CopyCat.…”

Section: Gamesmentioning

confidence: 99%

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

et al. 2011

View full text Add to dashboard Cite

Human hand recognition plays an important role in a wide range of applications ranging from sign language translators, gesture recognition, augmented reality, surveillance and medical image processing to various Human Computer Interaction (HCI) domains. Human hand is a complex articulated object consisting of many connected parts and joints. Therefore, for applications that involve HCI one can find many challenges to establish a system with high detection and recognition accuracy for hand posture and/or gesture. Hand posture is defined as a static hand configuration without any movement involved. Meanwhile, hand gesture is a sequence of hand postures connected by continuous motions. During the past decades, many approaches have been presented for hand posture and/or gesture recognition. In this paper, we provide a survey on approaches which are based on Hidden Markov Models (HMM) for hand posture and gesture recognition for HCI applications.

show abstract

Section: Gamesmentioning

confidence: 99%

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Before the MKST was available, body pose tracking with the Kinect camera was possible using the OpenNI framework and the PrimeSense NITE middleware. Zafrulla et al [5] used the PrimeSense tracker to perform sentence recognition on a dataset of ASL sentences from an educational game called CopyCat [26]. Copper et al [27] used the PrimeSense tracker to perform linguistic sub-unit-based sign language recognition on signs from German Sign Language.…”

Section: Related Workmentioning

confidence: 99%

Hand detection in American Sign Language depth data using domain-driven random forest regression

Zafrulla

Sahni

Bedri

et al. 2015

2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)

Self Cite

View full text Add to dashboard Cite

In Automatic Sign Language Recognition (ASLR), robust hand tracking and detection is key to good recognition accuracy. We introduce a new dataset of depth data from continuously signed American Sign Language (ASL) sentences. We present analysis showing numerous errors of the Microsoft Kinect Skeleton Tracker (MKST) in cases where hands are close to the body, close to each other, or when the arms cross. We also propose a method based on domain-driven random forest regression, which predicts real world 3D hand locations using features generated from depth images. We show that our hand detector (DDRFR) has >20% improvement over the MKST within a margin of error of 5 cm from the ground truth.

show abstract

“…Translation is an interesting problem because it also involves video rather than audio processing. [5] classify signs to verify the correctness of a child's response to a sign language tutoring program. Signs are classified using a forward and backward pass model, essentially considering possible phrases both forward and backward, using a confidence model and accelerometer data from gloves worn on the signers' hands.…”

Section: Relevance To Others' Workmentioning

confidence: 99%

“…Signs are classified using a forward and backward pass model, essentially considering possible phrases both forward and backward, using a confidence model and accelerometer data from gloves worn on the signers' hands. This work is particularly relevant since it also involves children but the intent in [5] is to teach sign language while our demonstration fosters discussion about technology to teach in sign language.…”

Section: Relevance To Others' Workmentioning

confidence: 99%

Owl pellets and head-mounted displays

Jones

Lawler

Hintz

et al. 2013

Proceedings of the 12th International Conference on Interaction Design and Children

View full text Add to dashboard Cite

This demonstration will provoke discussion of the role of head mounted displays (HMD) in Deaf science education for children. The demonstration mimics the classroom laboratory experience of children who are Deaf or hard-of-hearing. When these children dissect an owl pellet, their teachers can not stand behind them and offer instruction over their shoulder while the students looks at the specimen. A teacher can sign for the student but this requires the student to switch visual attention back and forth between the specimen and the signing teacher. This can be difficult if the specimen is on a table and the teacher is standing nearby. HMDs allow students to put the signing teacher and the laboratory specimen in close visual proximity. Participants in our demonstration will be given an owl pellet study kit and no verbal instruction. Participants will be asked to use visual aids to identify bones in the pellet. Some participants will view the visual aids on a poster placed behind them. Others will view visual aids in an HMD.

show abstract

American Sign Language Phrase Verification in an Educational Game for Deaf Children

Cited by 35 publications

References 9 publications

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

Hidden Markov model for human to computer interaction: a study on human hand gesture recognition

Hand detection in American Sign Language depth data using domain-driven random forest regression

Owl pellets and head-mounted displays

Contact Info

Product

Resources

About