Comparative Study of Visual Feature for Bimodal Hindi Speech Recognition

Abstract

In building speech recognition based applications, robustness to different noisy background condition is an important challenge. In this paper bimodal approach is proposed to improve the robustness of Hindi speech recognition system. Also an importance of different types of visual features is studied for audio visual automatic speech recognition (AVASR) system under diverse noisy audio conditions. Four sets of visual feature based on Two-Dimensional Discrete Cosine Transform feature (2D-DCT), Principal Component Analysis (PCA), Two-Dimensional Discrete Wavelet Transform followed by DCT (2D-DWT-DCT) and Two-Dimensional Discrete Wavelet Transform followed by PCA (2D-DWT-PCA) are reported. The audio features are extracted using Mel Frequency Cepstral coefficients (MFCC) followed by static and dynamic feature. Overall, 48 features, i.e. 39 audio features and 9 visual features are used for measuring the performance of the AVASR system. Also, the performance of the AVASR using noisy speech signal generated by using NOISEX database is evaluated for different Signal to Noise ratio (SNR: 30 dB to −10 dB) using Aligarh Muslim University Audio Visual (AMUAV) Hindi corpus. AMUAV corpus is Hindi continuous speech high quality audio visual databases of Hindi sentences spoken by different subjects.

Pattern Recognition Bird Sounds Based on Their Type Using Discreate Cosine Transform (DCT) and Gaussian Methods

“…Mel Filter Bank is used to convert sound signals in the frequency domain to the frequency domain mel and shows several energy quantities in the frequency range available in each filter mel. The approach in calculating mel is in frequency f (Hz) in Equation 4 [8].…”

“…The researcher removes the DCT coefficient value to zero even though it indicates the value of the frame signal [8]. Elimination of zero costs because previous studies conducted are not reliable on speech recognition [10].…”

An Assessment of the Visual Features Extractions for the Audio-Visual Speech Recognition

“…The greater part of the systems utilized in this sort of highlights uses hued lips or other stamping [6], anyway this methodology is a long way from this present reality circumstances. Procedures for programmed lip form extraction have been proposed by a few creators, however to-date these have met with restricted achievement [7] and an elective methodology is to receive appearance based features extractions strategies, which can be apply reasonable change of the mouth region of interest pursued by dimensionality decrease systems, for example linear discriminant analysis as well as principle component-analysis [8]. This article writes about a correlation of visual highlights separated utilizing an appearance methodology.…”