Uncertainty-based learning of acoustic models from noisy data

Abstract

Revised version including a bugfix in the computation of the Wiener uncertainty estimator and in the corresponding numerical results in Tables 1, 2, 3, E.6, E.7 and in Figure 6 compared to the original version published by Elsevier.International audienceWe consider the problem of acoustic modeling of noisy speech data, where the uncertainty over the data is given by a Gaussian distribution. While this uncertainty has been exploited at the decoding stage via uncertainty decoding, its usage at the training stage remains limited to static model adaptation. We introduce a new Expectation Maximisation (EM) based technique, which we call uncertainty training, that allows us to train Gaussian mixture models (GMMs) or hidden Markov models (HMMs) directly from noisy data with dynamic uncertainty. We evaluate the potential of this technique for a GMM-based speaker recognition task on speech data corrupted by real-world domestic background noise, using a state-of-the-art signal enhancement technique and various uncertainty estimation techniques as a front-end. Compared to conventional training, the proposed training algorithm results in 1% to 2% absolute improvement in speaker recognition accuracy by training from either matched, unmatched or multi-condition noisy data. This algorithm is also applicable with minor modifications to maximum a posteriori (MAP) or maximum likelihood linear regression (MLLR) acoustic model adaptation from noisy data and to other data than audio

Variational Bayesian Inference for Source Separation and Robust Feature Extraction

“…2) Vector Taylor Series: VTS consists of linearizing the MFCC transform by its first-order Taylor series expansion [43]. The mean of the features are computed as given in the MFCC computation.…”

“…Classification 1) Data and Algorithmic Settings: As an example classification task, we consider the noise-robust speaker identification 1-un-un 2-un-un 1-co-un 2-co-un 1-un-co 2-un-co 1-co-co 2-co-co ML [11] 1.58 benchmark in [43]. This benchmark consists of noiseless reverberated utterances and real domestic noise backgrounds from the 2nd CHiME Speech Separation and Recognition Challenge [55] which are mixed together at six different signal-to-noise ratios (SNR) from -6 to 9 dB.…”

“…Finally, speaker identification is achieved using 32-component Gaussian mixture models (GMM) trained on the noiseless reverberated utterances of the training set. As argued in [43], this benchmark was designed to assess the accuracy of the estimated uncertainties alone without interference from more complex classifiers involved in state-of-the-art speaker identification systems. For more details about the data, the configuration of FASST, and the classifier, please refer to [43].…”

“…Uncertainty propagation has emerged as a promising complementary approach whereby the separated target signal and its features are not considered as point estimates anymore but their posterior distribution is approximated as a Gaussian with time-varying variance or uncertainty that is exploited by the classifier [3], [4]. This approach was introduced for noise-robust automatic speech recognition [36]- [42] and it has also been used for noiserobust speaker identification [43], [44] and singer identification in polyphonic music [45]. While there exist techniques to propagate uncertainty from the separated signal to the features based on moment matching [46], unscented transform [38], or Vector Taylor series (VTS) [47], the estimation of uncertainty on the separated signal remains a difficult problem.…”

An extended experimental investigation of DNN uncertainty propagation for noise robust ASR

“…The distribution of speech distortions is typically approximated as a Gaussian from which the uncertainty or variance of speech distortions is derived. The uncertainty can be computed directly in the ASR feature domain [2,[6][7][8][9][10] or propagated from the spectral domain to the feature domain [1,[11][12][13][14][15].…”