A Support Vector Machine-Based Dynamic Network for Visual Speech Recognition Applications

Gordan, Mihaela; Kotropoulos, Constantine; Pitas, Ioannis

doi:10.1155/s1110865702207039

Cited by 35 publications

(20 citation statements)

References 27 publications

(33 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speaker diarization is the problem of determining "who spoke when" based on audio signals [Anguera Miro et al 2012]. The problem of speech recognition based on visual information has been studied by Gordan et al [2002] and Saenko et al [2004;. Visual and audio data are often used together in speaker localization ( [Nock et al 2003;Potamianos et al 2003]).…”

Section: Related Workmentioning

confidence: 99%

Speaker-Following Video Subtitles

Kautz

et al. 2015

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

We propose a new method for improving the presentation of subtitles in video (e.g. TV and movies). With conventional subtitles, the viewer has to constantly look away from the main viewing area to read the subtitles at the bottom of the screen, which disrupts the viewing experience and causes unnecessary eyestrain. Our method places on-screen subtitles next to the respective speakers to allow the viewer to follow the visual content while simultaneously reading the subtitles. We use novel identification algorithms to detect the speakers based on audio and visual information. Then the placement of the subtitles is determined using global optimization. A comprehensive usability study indicated that our subtitle placement method outperformed both conventional fixed-position subtitling and another previous dynamic subtitling method in terms of enhancing the overall viewing experience and reducing eyestrain.

show abstract

Section: Related Workmentioning

confidence: 99%

Speaker-Following Video Subtitles

Kautz

et al. 2015

ACM Trans. Multimedia Comput. Commun. Appl.

View full text Add to dashboard Cite

show abstract

“…Moreover, by using a method common to both the audio and visual aspects of speech, there is the potential for a more straightforward combination of results obtained from separate audio and visual investigations and such integration has often been carried out using machine learning techniques, such as time delay neural network (TDNN) [42], support vector machines (SVM) [43] and AdaBoost [44].…”

Section: Speech Classification Based On Lip Featuresmentioning

confidence: 99%

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Ibrahim

Mulvaney

2015

Journal of Visual Communication and Image Representation

View full text Add to dashboard Cite

“…We used the 450 TIMIT-SX sentences originally designed to provide a good coverage of phonetic contexts of the English language in as few words as possible [17]. Each speaker was asked to read 20 sentences.…”

Section: A11 Linguistic Contentmentioning

confidence: 99%

“…The acoustic models for the forced-path alignment pro-cess were seeded from models generated from the TIMIT corpus [17]. Because the noise level of the AV-TIMIT corpus was higher than that of TIMIT (which was recorded with a noise-canceling closetalking microphone), the initial time-aligned transcriptions were not as accurate as we had desired (as determined by expert visual inspection against spectrograms).…”

Section: A21 Audio Processingmentioning

confidence: 99%

Articulatory features for robust visual speech recognition

Saenko

Darrell

Glass

2004

Proceedings of the 6th International Conference on Multimodal Interfaces

View full text Add to dashboard Cite

This thesis explores a novel approach to visual speech modeling. Visual speech, or a sequence of images of the speaker's face, is traditionally viewed as a single stream of contiguous units, each corresponding to a phonetic segment. These units are defined heuristically by mapping several visually similar phonemes to one visual phoneme, sometimes referred to as a viseme. However, experimental evidence shows that phonetic models trained from visual data are not synchronous in time with acoustic phonetic models, indicating that visemes may not be the most natural building blocks of visual speech. Instead, we propose to model the visual signal in terms of the underlying articulatory features. This approach is a natural extension of feature-based modeling of acoustic speech, which has been shown to increase robustness of audiobased speech recognition systems. We start by exploring ways of defining visual articulatory features: first in a data-driven manner, using a large, multi-speaker visual speech corpus, and then in a knowledge-driven manner, using the rules of speech production. Based on these studies, we propose a set of articulatory features, and describe a computational framework for feature-based visual speech recognition. Multiple feature streams are detected in the input image sequence using Support Vector Machines, and then incorporated in a Dynamic Bayesian Network to obtain the final word hypothesis. Preliminary experiments show that our approach increases viseme classification rates in visually noisy conditions, and improves visual word recognition through feature-based context modeling.

show abstract

A Support Vector Machine-Based Dynamic Network for Visual Speech Recognition Applications

Cited by 35 publications

References 27 publications

Speaker-Following Video Subtitles

Speaker-Following Video Subtitles

Geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping

Articulatory features for robust visual speech recognition

Contact Info

Product

Resources

About