Data Balancing for Efficient Training of Hybrid ANN/HMM Automatic Speech Recognition Systems

Abstract

Abstract-Hybrid speech recognizers, where the estimation of the emission pdf of the states of Hidden Markov Models (HMMs), usually carried out using Gaussian Mixture Models (GMMs), is substituted by Artificial Neural Networks (ANNs) have several advantages over the classical systems. However, to obtain performance improvements, the computational requirements are heavily increased because of the need to train the ANN.Departing from the observation of the remarkable skewness of speech data, this paper proposes sifting out the training set and balancing the amount of samples per class. With this method the training time has been reduced 18 times while obtaining performances similar to or even better than those with the whole database, especially in noisy environments.However, the application of these reduced sets is not straightforward. To avoid the mismatch between training and testing conditions created by the modification of the distribution of the training data, a proper scaling of the a posteriori probabilities obtained and a resizing of the context window need to be performed as demonstrated in the paper.

Real-Time Robust Automatic Speech Recognition Using Compact Support Vector Machines

“…The problem of obtaining scaled likelihoods from a posteriori probabilities in the hybrid ANN/HMM context was an open issue since mismatches between the a priori probabilities of the training and test databases led to inconsistent results [1], [4], [20], [63], [65], [66]. In [5] it is shown that scaled likelihoods should always be estimated using the prior probabilities from the training data. In our case, the balancing of the training set enables the interpretation of the outputs of the SVM as scaled likelihoods without the need of applying any corrections.…”

“…In this work, the first problem is alleviated by randomly selecting a balanced subset of training samples, which significantly reduces the computational cost of the training process while causing negligible reduction in performance, as it was previously demonstrated in the ANN/HMM paradigm [5]. However, further research on this issue is required for the proposed system to manage more demanding ASR tasks.…”

“…However, the HMM-based ASR systems seem to be close to reaching their limit of performance. Hybrid systems based on a combination of artificial neural networks (ANNs) and HMMs, referred to as hybrid ANN/HMM [1]- [3], provide significant performance improvements in noisy conditions [4], [5]. However, progress on this paradigm has been hindered by their training computational requirements, A. I. García-Moral is with Fonetic Solutions S. L., Madrid -28037, Spain (e-mail: ana.garcia@fonetic.es).…”

A Comparitive Survey of ANN and Hybrid HMM/ANN Architectures for Robust Speech Recognition

“…The learning algorithm can be the conventional backpropagation [18], or a more sophisticated variation of it [2]. In the learning phase, the desired output is 1 for the correct and 0 for all other speech units.…”

“…This important research finding eased their integration with the HMM current state of the art recognition system technology [2]. This fact led to the possibility of unifying HMM and ANN within unifying novel models [5].…”