Visual Speech Recognition: A Deep Learning Approach

Mudaliar, Navin Kumar; Hegde, Kavita; Ramesh, Anand; Patil, Varsha

doi:10.1109/icces48766.2020.9137926

Cited by 12 publications

(4 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Accurate recognition of spoken words enhances the system's user experience, reliability, and productivity, making it an essential consideration when designing and evaluating VSR systems. Hence, the right combination of DL architectures and datasets greatly improves the VSR system, which is visible in [161], [163].…”

Section: E the Vital Combination Of DL Architecture And Proper Datasetsmentioning

confidence: 98%

“…A customized dataset with more than 48,000 images was used to evaluate the lip segmentation using this model and a testing accuracy of 98.4% was depicted. Navin et al [163] proposed a deep learning model for VSR to perform word-level classification. ResNet architecture is used along with 3D convolution layers and Gated recurrent units (GRU).…”

Section: Sequence Of Video Framesmentioning

confidence: 99%

See 1 more Smart Citation

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Nemani¹,

Krishna²,

Ramisetty³

et al. 2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Speaker-independent visual speech recognition (VSR) is a complex task that involves identifying spoken words or phrases from video recordings of a speaker's facial movements. Decoding the intricate visual dynamics of a speaker's mouth in a high-dimensional space is a significant challenge in this field. To address this challenge, researchers have employed advanced techniques that enable machines to recognize human speech through visual cues automatically. Over the years, there has been a considerable amount of research in the field of VSR involving different algorithms and datasets to evaluate system performance. These efforts have resulted in significant progress in developing effective VSR models, creating new opportunities for further research in this area. This survey provides a detailed examination of the progression of VSR over the past three decades, with a particular emphasis on the transition from speaker-dependent to speaker-independent systems. We also provide a comprehensive overview of the various datasets used in VSR research and the preprocessing techniques employed to achieve speaker independence. The survey covers the works published from 1990 to 2023, thoroughly analyzing each work and comparing them on various parameters. This survey provides an in-depth analysis of speaker-independent VSR systems evolution from 1990 to 2023. It outlines the development of VSR systems over time and highlights the need to develop end-to-end pipelines for speakerindependent VSR. The pictorial representation offers a clear and concise overview of the techniques used in speaker-independent VSR, thereby aiding in the comprehension and analysis of the various methodologies. The survey also highlights the strengths and limitations of each technique and provides insights into developing novel approaches for analyzing visual speech cues. Overall, This comprehensive review provides insights into the current state-of-the-art speaker-independent VSR and highlights potential areas for future research.

show abstract

Section: E the Vital Combination Of DL Architecture And Proper Datasetsmentioning

confidence: 98%

Section: Sequence Of Video Framesmentioning

confidence: 99%

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Nemani¹,

Krishna²,

Ramisetty³

et al. 2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

show abstract

“…Visual-only approach, where spoken word or letter is decided solely based on visual cues, spanned across multiple languages, such as English [8], Japanese [9] and Bahasa Indonesia [10]. For Bahasa Indonesia, by focusing on the nature of lip motion acting as a visual cue of the speaker, Maxalmina concluded that it is possible to detect specific vowels with the highest accuracy of 84%.…”

Section: Introductionmentioning

confidence: 99%

Facial Movement Recognition Using CNN-BiLSTM in Vowel for Bahasa Indonesia

Rahman,

Wicaksono,

Yuniarno

et al. 2024

JAREE

View full text Add to dashboard Cite

Speaking is a multimodal phenomenon that has both verbal and non-verbal cues. One of the non-verbal cues in speaking is the facial movement of the subject, which can be used to find the letter being spoken by the subject. Previous research has been done to prove that lip movement can translate to vowels for Bahasa Indonesia, but detecting the whole facial movement is yet to be covered. This research aimed to establish a CNN-BiLSTM model that can learn spoken vowels by reading the subject's facial movements. The CNN-BiLSTM model yielded a 98.66% validation accuracy, with over 94% accuracy for all five vowels. The model is also capable of recognizing whether the subject is currently silent or speaking a vowel with 98.07% accuracy.

show abstract

“…Another internal element that might cause intra-speaker vocal fluctuation is emotion [6]. Emotion identification is the process for detecting and identifying emotions.…”

mentioning

confidence: 99%

Artificial and Deep Knowledge Spiritual Speaker Recognition

2023

IJBPAS

View full text Add to dashboard Cite

A method for recognising a speaker based on speech features is known as speaker recognition. Speaker recognition technology is frequently employed in a variety of fields. Most speaker identification algorithms have been developed on regular, clean recordings, but their effectiveness degrades when hearing speech with emotions. This study offers an emotional speech signals system developed using standard machine learning algorithms on an emotional speech database obtained from the University Audio visual Database of Affective Speech and Song using temporal, frequency, and spectral properties Five models (Logistic Regression, Support Vector Machine, Variational Forest, XGBoost, and k-Nearest Neighbor) and three deep learning models (Logistic Regression, Logistic Regression, Random Forest, XGBoost, and k-Nearest Neighbor) were trained and compared in terms of performance (Long Short-Term Memory network, Multilayer Perceptron, and Convolutional Neural Network).Deep neural networks outperformed state-of-the-art models

show abstract

Visual Speech Recognition: A Deep Learning Approach

Cited by 12 publications

References 10 publications

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Facial Movement Recognition Using CNN-BiLSTM in Vowel for Bahasa Indonesia

Artificial and Deep Knowledge Spiritual Speaker Recognition

Contact Info

Product

Resources

About