Rapid signer adaptation for continuous sign language recognition using a combined approach of eigenvoices, MLLR, and MAP

Agris, U. von; Blomer, C.; Kraiss, Karl-Friedrich

doi:10.1109/icpr.2008.4761363

Cited by 22 publications

(8 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The problem of signer adaptation has been addressed in prior work, using techniques borrowed from speaker adaptation for speech recognition, such as maximum likelihood linear regression and maximum a posteriori estimation [14,67,68], and to explicitly study the contrast between signer-dependent and multi-signer fingerspelling recognition.…”

Section: Related Workmentioning

confidence: 99%

Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation

Kim

Keane

Wang

et al. 2017

Computer Speech & Language

View full text Add to dashboard Cite

We study the problem of recognizing video sequences of fingerspelled letters in American Sign Language (ASL). Fingerspelling comprises a significant but relatively understudied part of ASL. Recognizing fingerspelling is challenging for a number of reasons: It involves quick, small motions that are often highly coarticulated; it exhibits significant variation between signers; and there has been a dearth of continuous fingerspelling data collected. In this work we collect and annotate a new data set of continuous fingerspelling videos, compare several types of recognizers, and explore the problem of signer variation. Our best-performing models are segmental (semiMarkov) conditional random fields using deep neural network-based features. In the signer-dependent setting, our recognizers achieve up to about 92% letter accuracy. The multi-signer setting is much more challenging, but with neural network adaptation we achieve up to 83% letter accuracies in this setting.

show abstract

Section: Related Workmentioning

confidence: 99%

Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation

Kim

Keane

Wang

et al. 2017

Computer Speech & Language

View full text Add to dashboard Cite

show abstract

“…With 80 and 160 labeled signs, they achieved 78.6% and 94.6% accuracy respectively on a vocabulary of 153 signs. In their latest work [6], they proposed the eigenvoice + MLLP + MAP approach. Wang et al [7] presented a supervised adaptive method based on data generating, in which they reduced the size of adaptation data set from 350 to 136 with an acceptable recognition rate.…”

Section: Introductionmentioning

confidence: 99%

“…Adapt each CV model set with its A data set (6). If the number of maximum iteration does not arrive, go to step 7, or else go to step 8.…”

mentioning

confidence: 99%

Hypothesis comparison guided cross validation for unsupervised signer adaptation

Zhou

Yang

Lin

et al. 2011

2011 IEEE International Conference on Multimedia and Expo

View full text Add to dashboard Cite

Signer adaptation is important to sign language recognition systems in that a one-size-fits-all model set can not perform well on all kinds of signers. Supervised signer adaptation must utilize the labeled adaptation data that are collected explicitly. To skip the data collecting process in signer adaptation, we propose an unsupervised adaptation method called hypothesis comparison guided cross validation (HC-CV) algorithm. The algorithm not only addresses the problem of overlap between the data set to be labeled and the data set for adaptation, but also employs an additional hypothesis comparison step to decrease the noise rate of the adaptation data set. Experimental results show that the HC-CV adaptation algorithm is superior to the CV adaptation algorithm and the conventional self-teaching algorithm. Though the algorithm is proposed for signer adaptation, it can also be applied to speaker adaptation and writer adaptation straightforwardly.

show abstract

“…In [3] Von Agris et al presented a comprehensive SLR system using techniques from speech recognition to adapt the signer features and classification, making the recognition task signer independent. In other work [1], they demonstrated how three approaches to speaker adaptation in speech recognition can be successfully applied to the problem of signer adaptation for signer independent sign language recognition. They contrasted a PCA based approach, a maximum likelihood linear regression approach and a maximum a posteriori probability (MAP) estimation approach, and finally showed how they can be combined to yield superior results .…”

Section: Signer Independencementioning

confidence: 99%

“…1 These encode different elements of a sign. Unlike speech they do not have to occur sequentially, but can be combined in parallel to describe a sign.…”

Section: Sign Linguisticsmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

182

View full text Add to dashboard Cite

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data sets.

show abstract

Rapid signer adaptation for continuous sign language recognition using a combined approach of eigenvoices, MLLR, and MAP

Cited by 22 publications

References 6 publications

Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation

Lexicon-free fingerspelling recognition from video: Data, models, and signer adaptation

Hypothesis comparison guided cross validation for unsupervised signer adaptation

Sign Language Recognition

Contact Info

Product

Resources

About