Improving connected letter recognition by lipreading

Bregler, Christoph; Hild, Hermann; Manke, S.; Waibel, Alexander

doi:10.1109/icassp.1993.319179

Cited by 93 publications

(70 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As the application domain is the same, lip reading classification techniques are often the same as those applied in the audio speech recognition (ASR) field and, consequently, dynamic time warping (DTW) [39,40] and HMMs [10,18,41], are popular. Moreover, by using a method common to both the audio and visual aspects of speech, there is the potential for a more straightforward combination of results obtained from separate audio and visual investigations and such integration has often been carried out using machine learning techniques, such as time delay neural network (TDNN) [42], support vector machines (SVM) [43] and AdaBoost [44].…”

Section: Speech Classification Based On Lip Featuresmentioning

confidence: 99%

“…DTW utilizes dynamic programming to generate candidate stretched and compressed sections in sequences of feature vectors, in order to find an alignment between two time-series that minimizes distortion [57] and in doing so produces a suitable warping function that minimizes the total distance (normally Euclidean) between an unknown sample and the reference template; While a DTW-based lip reading system has been proposed previously [39,40,58], to the best of the authors' knowledge the problem has not been addressed using multi-dimensional DTW and reference template probabilities.…”

Section: Template Probability Multi Dimensional Dynamic Time Warpingmentioning

confidence: 99%

See 1 more Smart Citation

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Ibrahim

Mulvaney

2015

Journal of Visual Communication and Image Representation

View full text Add to dashboard Cite

Section: Speech Classification Based On Lip Featuresmentioning

confidence: 99%

Section: Template Probability Multi Dimensional Dynamic Time Warpingmentioning

confidence: 99%

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Ibrahim

Mulvaney

2015

Journal of Visual Communication and Image Representation

View full text Add to dashboard Cite

“…In the previous study [5] In this paper [6], the system proposed by Ralph Krike et al operates under the influence of active near infrared illumination. It makes use of a technique which makes use of local binary patterns, to model lip motions with hidden Markov models.…”

Section: Figure 2: Appearance Of Lip During Various Soundmentioning

confidence: 99%

Implication and Utilization of various Lip Reading Techniques

Khan¹,

Azmi²,

Nair³

et al. 2017

IJCA

View full text Add to dashboard Cite

Ears are one of the most important sensory organs of the human body. It is one of our primary sensors along with our eyes, nose, skin etc. However, while many people may have perfectly functional ears some suffer from partial or total hearing loss or hearing impairment. In order to tackle this, many suggest the use of hearing aids, sign language, cochlear implants. But of all the different ways, one of the best way to beat this inability is to train oneself to lip read. In simple words, lip reading is the interpretation of the movements of not only the lips but also the face and tongue. While many people are excellent at lip reading, some find it a challenge too difficult to overcome. In regard to this, one cannot deny that whenever some human falls short of something, a manmade technology can make up for the shortcoming to a certain extent. As a result, over the years a lot of time and effort has been invested in studying various lip reading algorithms and speech automations. Through this research paper, the aim is to put forth a study on various lip-reading algorithms, their approaches, their innovations and the central ideas behind their proposed studies.

show abstract

“…The solution we propose is based on this approach, but contrarily to published works, e.g. [1,4], our systems deal with abovementioned phenomena of temporal shift between audio and visual sources.…”

Section: Integration Of Visual Informationmentioning

confidence: 99%

Asynchronous integration of visual information in an automatic speech recognition system

Alissali

Deléglise²,

Rogozan³

Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96

View full text Add to dashboard Cite

This paper deals with the integration of visual data in automatic speech recognition systems. We rst describe the framework of our research; the development of advanced multi-user multi-modal interfaces. Then we present audiovisual speech recognition problems in general, and the ones we are interested in, in particular. After a very brief discussion of existing systems, the major part of the paper describes the systems we developed according to two dierent approaches to the problem of integration of visual data in speech recognition systems.Section 3 presents the architecture of our audio-only reference and baseline systems. Our audio-visual systems are described in Section 2. We rst describe a system we developed according to the rst approach (called the direct integration model) and show its limitations. Our approach, which w e call asynchronous integration, is then presented in Sectio 4.2. After the general guidelines, we g o i n to some details about the distributed architecture and the variant o f the N-best algorithm we developed for the implementation of this approach.In Section 6 the performances of these dierent systems are compared, then we conclude by a brief discussion of the performance improvements we obtain and future work.

show abstract

Improving connected letter recognition by lipreading

Cited by 93 publications

References 9 publications

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Implication and Utilization of various Lip Reading Techniques

Asynchronous integration of visual information in an automatic speech recognition system

Contact Info

Product

Resources

About