A Comparison of Model Validation Techniques for Audio-Visual Speech Recognition

Abstract

This paper implements and compares the performance of a number of techniques proposed for improving the accuracy of Automatic Speech Recognition (ASR) systems. As ASR that uses only speech can be contaminated by environmental noise, in some applications it may be improve performance to employ Audio-Visual Speech Recognition (AVSR), in which recognition uses both audio information and mouth movements obtained from a video recording of the speaker's face region. In this paper, model validation techniques, namely the holdout method, leave-one-out cross validation and bootstrap validation, are implemented to validate the performance of an AVSR system as well as to provide a comparison of the performance of the validation techniques themselves. A new speech data corpus is used, namely the Loughborough University Audio-Visual (LUNA-V) dataset that contains 10 speakers with five sets of samples uttered by each speaker. The database is divided into training and testing sets and processed in manners suitable for the validation techniques under investigation. The performance is evaluated using a range of different signal-to-noise ratio values using a variety of noise types obtained from the NOISEX-92 dataset.

WADA-W: A Modified WADA SNR Estimator for Audio-Visual Speech Recognition

“…All the experiments result used the 'leave-one-out' cross validation (LOOCV) method to cross check the result. Based on previous research work [25], LOOCV techniques have been able to achieve a slightly improved accuracy compared to the more common Holdout and bootstrap validation approach. Furthermore, the LOOCV technique is able to allow an evaluation of every sample and obtain the final accuracy value by averaging the results from all samples.…”

An intelligent voice recognition system based on fuzzy logic and the bag-of-words technique

“…IOP Publishing doi:10.1088/1757-899X/1230/1/012020 2 HMMs allow to perform preliminary formation of a reference database of words or phrases and then to identify the presented sound images based on comparison with this database [6,7]. One of the types of HMMs is continuous HMMs, where transitions between hidden states and the arrival of observations can occur at arbitrary points of time [8,9].…”