Sign Language Recognition, Generation, and Modelling: A Research Effort with Applications in Deaf Communication

Efthimiou, Eleni; Fotinea, Stavroula–Evita; Vogler, Christian; Hanke, Thomas; Glauert, John; Bowden, Richard; Braffort, Annelies; Collet, Christophe; Maragos, Petros; Segouat, Jérémie

doi:10.1007/978-3-642-02707-9_3

Cited by 58 publications

(38 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Despite these problems recent uses of SLR include translation to spoken language, or to another sign language when combined with avatar technology [3,25]. Sign video data once recognised can be compressed using SLR into an encoded form (e.g.…”

Section: Discussionmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

Self Cite

182

View full text Add to dashboard Cite

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data sets.

show abstract

Section: Discussionmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

Self Cite

182

View full text Add to dashboard Cite

show abstract

“…A pool of words or sentences used in traffic offices when renewing the driving license was the field of study in (Efthimiou et al, 2009); another related to bus transportation info was studied in (Efthimiou et al, 2012). Many challenges exist while converting spoken Arabic to ASL.…”

Section: Spoken Language To Animationmentioning

confidence: 99%

An Enhanced Training- Based Arabic Sign Language Virtual Interpreter Using Parallel Recurrent Neural Networks

Abdou¹

2018

Journal of Computer Science

View full text Add to dashboard Cite

Intelligent machine translation systems have a remarkable importance in integrating people with disabilities in community. Arabic to Arabic sign language systems are limited. Deep Learning (DL) was successfully applied to problems related to music information retrieval, image recognition and text recognition, but its use in sign language recognition is rare. This paper introduces an automatic virtual translation system from Arabic language into Arabic Sign Language (ASL) via a popular DL architecture: The Recurrent Neural Network (RNN). The proposed system uses a deep neural network training-based system for ASL that convolves RNN and Graphical Processing Unit (GPU) parallel processors. The system is evaluated using both objective and subjective measures. Obtained results are towards reducing errors, speeding up avatar and expressing signs and facial expressions in a well-received manner by Deaf. The signing avatar is highly encouraged as a simulator for natural human signs.

show abstract

“…There are very few publicly available datasets that are suitable for the training of such recognition systems, e.g. [20], [5].…”

Section: Mouth Non-manuals For Weakly Supervised Aslr Trainingmentioning

confidence: 99%

A survey on mouth modeling and analysis for Sign Language recognition

Antonakos

Roussos

Zafeiriou

2015

2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)

View full text Add to dashboard Cite

Abstract-Around 70 million Deaf worldwide use Sign Languages (SLs) as their native languages. At the same time, they have limited reading/writing skills in the spoken language. This puts them at a severe disadvantage in many contexts, including education, work, usage of computers and the Internet. Automatic Sign Language Recognition (ASLR) can support the Deaf in many ways, e.g. by enabling the development of systems for Human-Computer Interaction in SL and translation between sign and spoken language. Research in ASLR usually revolves around automatic understanding of manual signs. Recently, ASLR research community has started to appreciate the importance of non-manuals, since they are related to the lexical meaning of a sign, the syntax and the prosody. Nonmanuals include body and head pose, movement of the eyebrows and the eyes, as well as blinks and squints. Arguably, the mouth is one of the most involved parts of the face in non-manuals. Mouth actions related to ASLR can be either mouthings, i.e. visual syllables with the mouth while signing, or non-verbal mouth gestures. Both are very important in ASLR. In this paper, we present the first survey on mouth non-manuals in ASLR. We start by showing why mouth motion is important in SL and the relevant techniques that exist within ASLR. Since limited research has been conducted regarding automatic analysis of mouth motion in the context of ALSR, we proceed by surveying relevant techniques from the areas of automatic mouth expression and visual speech recognition which can be applied to the task. Finally, we conclude by presenting the challenges and potentials of automatic analysis of mouth motion in the context of ASLR.

show abstract

Sign Language Recognition, Generation, and Modelling: A Research Effort with Applications in Deaf Communication

Cited by 58 publications

References 16 publications

Sign Language Recognition

Sign Language Recognition

An Enhanced Training- Based Arabic Sign Language Virtual Interpreter Using Parallel Recurrent Neural Networks

A survey on mouth modeling and analysis for Sign Language recognition

Contact Info

Product

Resources

About