Lip Shape and Hand Position Fusion for Automatic Vowel Recognition in Cued Speech for French

Heracleous, Panikos; Aboutabit, Noureddine; Beautemps, Denis

doi:10.1109/lsp.2009.2016011

Cited by 29 publications

(20 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous studies (e.g., [4,5] the authors used a video processing technique based on blue color in order to track the hand positions and handshapes. In this study, landmarks with different colors were placed on the fingers resulting in a faster and more accurate image processing stage.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Cued Speech: A visual communication mode for the deaf society

Heracleous

Beautemps

2010

IEICE Electron. Express

View full text Add to dashboard Cite

Cued Speech is a visual mode of communication that uses handshapes and placements in combination with the mouth movements of speech to make the phonemes of a spoken language look different from each other and clearly understandable to deaf individuals. The aim of Cued Speech is to overcome the problems of lip reading and thus enable deaf persons to wholly understand spoken language. In this study, automatic phoneme recognition in Cued Speech for French based on hidden Markov model (HMMs) is introduced. The phoneme correct for a normal-hearing cuer was 82.9%, and for a deaf 81.5%. The results also showed, that creating cuer-independent HMMs should not face any specific difficulties, other than those occured in audio speech recognition.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Previously the authors presented vowel- [4], consonant- [5] and isolated word recognition [6] in Cued Speech for French based on HMMs. In the current study, continuous phoneme recognition is introduced using data from a deaf and normal-hearing cuer.…”

Section: Introductionmentioning

confidence: 99%

Cued Speech: A visual communication mode for the deaf society

Heracleous

Beautemps

2010

IEICE Electron. Express

View full text Add to dashboard Cite

show abstract

“…It is known that human lip readers rely heavily on context when lip reading and also have training tricks, which allow them to set a baseline for a new subject, such as asking them questions where the answers are either known or easily inferred. Heracleous et al showed that using the hand shapes from cued speech (where hand gestures are used to disambiguate vowels in spoken words for lip readers) improved the recognition rate of lip reading significantly [42]. They modelled the lip using some basic shape parameters, however it is also possible to track the lips, as shown by Ong and Bowden who use rigid flocks of linear predictors to track 34 points on the contour of the lips [81].…”

Section: Non-manual Featuresmentioning

confidence: 99%

Sign Language Recognition

Cooper

Holt

Bowden

2011

Visual Analysis of Humans

182

View full text Add to dashboard Cite

This chapter covers the key aspects of Sign Language Recognition (SLR), starting with a brief introduction to the motivations and requirements, followed by a précis of sign linguistics and their impact on the field. The types of data available and the relative merits are explored allowing examination of the features which can be extracted. Classifying the manual aspects of sign (similar to gestures) is then discussed from a tracking and non-tracking viewpoint before summarising some of the approaches to the non-manual aspects of sign languages. Methods for combining the sign classification results into full SLR are given showing the progression towards speech recognition techniques and the further adaptations required for the sign specific case. Finally the current frontiers are discussed and the recent research presented. This covers the task of continuous sign recognition, the work towards true signer independence, how to effectively combine the different modalities of sign, making use of the current linguistic research and adapting to larger more noisy data sets.

show abstract

“…Therefore, in 1967, Cornett de veloped the Cued Speech system as a supplement to lip read ing [2,3].…”

Section: Introductionmentioning

confidence: 99%

Automatic recognition of speech without any audio information

Heracleous¹,

Hagita²

2011

2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This article introduces automatic recognition of speech with out any audio information. Movements of the tongue, lips, and jaw are tracked by an Electro-Magnetic Articulogra phy (EMA) device and are used as features to create hid den Markov models (HMMs) and conduct automatic speech recognition in a conventional way. The results obtained are promising, which confirm that phonetic features characteriz ing articulation are as discriminating as those characterizing acoustics (except for voicing). The results also show that us ing tongue parameters result in a higher accuracy compared with the lip parameters.

show abstract

Lip Shape and Hand Position Fusion for Automatic Vowel Recognition in Cued Speech for French

Abstract: Abstract-Cued

Cited by 29 publications

References 8 publications

Cued Speech: A visual communication mode for the deaf society

Cued Speech: A visual communication mode for the deaf society

Sign Language Recognition

Automatic recognition of speech without any audio information

Contact Info

Product

Resources

About