SignStream: A tool for linguistic and computer vision research on visual-gestural language data

Neidle, Carol; Sclaroff, Stan; Athitsos, Vassilis

doi:10.3758/bf03195384

Cited by 63 publications

(36 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Manual transcription of such video data is time-consuming, and machine vision assisted annotation would greatly improve efficiency. Head tracking and handshape recognition algorithms [99], and sign word boundary detection algorithms [83] have been applied for this purpose. .…”

Section: Introductionmentioning

confidence: 99%

Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

Ong

Ranganath²

2005

IEEE Trans. Pattern Anal. Machine Intell.

454

217

View full text Add to dashboard Cite

Abstract-Research in automatic analysis of sign language has largely focused on recognizing the lexical (or citation) form of sign gestures as they appear in continuous signing, and developing algorithms that scale well to large vocabularies. However, successful recognition of lexical signs is not sufficient for a full understanding of sign language communication. Nonmanual signals and grammatical processes which result in systematic variations in sign appearance are integral aspects of this communication but have received comparatively little attention in the literature. In this survey, we examine data acquisition, feature extraction and classification methods employed for the analysis of sign language gestures. These are discussed with respect to issues such as modeling transitions between signs in continuous signing, modeling inflectional processes, signer independence, and adaptation. We further examine works that attempt to analyze nonmanual signals and discuss issues related to integrating these with (hand) sign gestures. We also discuss the overall progress toward a true test of sign recognition systems-dealing with natural signing by native signers. We suggest some future directions for this research and also point to contributions it can make to other fields of research. Web-based supplemental materials (appendicies) which contain several illustrative examples and videos of signing can be found at www.computer.org/publications/dlib.

show abstract

Section: Introductionmentioning

confidence: 99%

Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

Ong

Ranganath²

2005

IEEE Trans. Pattern Anal. Machine Intell.

454

217

View full text Add to dashboard Cite

show abstract

“…This publicly available corpus, including 15 short narratives plus hundreds of additional elicited utterances, includes multiple synchronized views of the signing (generally 2 stereoscopic front views plus a side view and a close-up of the face), which have been linguistically annotated using SignStream TM [15,17] software, which enables identification of the start and end points of the manual and non-manual components of the signing. From this corpus we selected training and testing sets of 32 and 13 video clips, respectively, of isolated utterances, extracting the segments containing non-manual markers of the classes of interest.…”

Section: Resultsmentioning

confidence: 99%

A Framework for the Recognition of Nonmanual Markers in Segmented Sequences of American Sign Language

Michael¹,

Yang²,

Liu³

et al. 2011

Procedings of the British Machine Vision Conference 2011

Self Cite

View full text Add to dashboard Cite

Despite the fact that there is critical grammatical information expressed through facial expressions and head gestures, most research in the field of sign language recognition has primarily focused on the manual component of signing. We propose a novel framework for robust tracking and analysis of non-manual behaviours, with an application to sign language recognition. The novelty of our method is threefold. First, we propose a dynamic feature representation. Instead of using only the features available in the current frame (e.g., head pose), we additionally aggregate and encode the feature values in neighbouring frames to better encode the dynamics of expressions and gestures (e.g., head shakes). Second, we use Multiple Instance Learning [12] to handle feature misalignment resulting from drifting of the face tracker and partial occlusions. Third, we utilize a discriminative Hidden Markov Support Vector Machine (HMSVM) [1] to learn finer temporal dependencies between the features of interest. We apply our signerindependent framework to segmented recognition of five classes of grammatical constructions conveyed through facial expressions and head gestures: wh-questions, negation, conditional/when clauses, yes/no questions and topics, and show improvement over previous methods.

show abstract

“…The subjects were native ASL signers. The annotations were carried out using SignStream TM , a database program to facilitate the analysis of visual language data [17]. Of particular relevance here were the annotations of positions and movements of the head and eyebrows, as well as the English translations provided for each sentence.…”

Section: Methodsmentioning

confidence: 99%

Facial Feature Tracking and Occlusion Recovery in American Sign Language

Castelli¹,

Betke²,

Neidle³

2006

6th International Workshop on Pattern Recognition in Information Systems

View full text Add to dashboard Cite

SignStream: A tool for linguistic and computer vision research on visual-gestural language data

Cited by 63 publications

References 23 publications

Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

Automatic Sign Language Analysis: A Survey and the Future beyond Lexical Meaning

A Framework for the Recognition of Nonmanual Markers in Segmented Sequences of American Sign Language

Facial Feature Tracking and Occlusion Recovery in American Sign Language

Contact Info

Product

Resources

About