The full combination sub-bands approach to noise robust HMM/ANN based ASR

Abstract

The performance of most ASR systems degrades rapidly with data mismatch relative to the data used in training. Under many realistic noise conditions a significant proportion of the spectral representation of a speech signal, which is highly redundant, remains uncorrupted. In the "missing feature" approach to this problem mismatching data is simply ignored, but the need to base recognition on unorthogonalised spectral features results in reduced performance in clean speech. In multiband ASR the results from independent recognition on a number of within-band orthogonalised sub-bands are combined. This approach more accurately reflects the uncertainty in mismatch detection, but loss of joint information due to independent sub-band processing can also result in reduced performance with clean speech. In this article the "full combination" approach to noise robust ASR is presented in which multiple data streams are associated not with individual sub-bands but with sub-band combinations. In this way no assumption of sub-band independence is required. Initial tests show some improved robustness to noise with no significant loss of performance with clean speech.

Confusion matrix based entropy correction in multi-stream combination

“…where the validation confusion probabilities matrix estimated using only frames detected as speech, and is estimated from all other frames. We used here a hard speech/non-speech decision based on the frame energy level and estimated noise energy alone (one could use a more sophisticated speech/non-speech detector) as follows, (9) if else (10) where in each frame is estimated as ( is the unused PLP coefficient), and is estimated as the average value over the first frames in each utterance (see Fig. 4).…”

“…Speech features used were cepstral domain PLP [6], excluding the energy coefficient "c0". Recognition was performed using the "all combinations multi-stream hybrid" model [9,10]. In this system a separate one hidden layer MLP is trained for each of the seven non-empty combinations of the three cepstral feature streams (PLP, delta PLP, delta delta PLP) (where delta PLP coefficients were over 9 windows, and delta delta over 9 windows).…”

“…In the context of multi-stream HMM/ANN speech recognition [1,9,10], the logic behind using the entropy in the output vector from each MLP classifier to weight the evidence supplied by that MLP [3,4,5,7,8,11,13] is that uncertain outputs with a flat distribution will have high entropy, while a confident peaked distribution will have low entropy. However, it has been observed that while MLP output entropy does usually increase with noise level, at high noise levels an MLP may also output a high probability that the noisy data comes from just the one or two "attractor" classes which happen to be closest to the noisy input in feature space.…”

A neural network for classification with incomplete data: application to robust ASR

“…Provided that the combined features input to each ANN expert are merely concatenated (i.e. no compression, orthogonalisation, or whatever is applied), the expected posteriors for each position of missing features can be computed directly from the IDCN parameters, and then simply combined in a linearly weighted sum [11] or geometrically weighted product [5].…”

“…If the position of missing data is not known, one successful approach [6,11,12] has been to train a separate classifier for each possible position of missing data and then to combine the posteriors for one class as a weighted sum over all classifiers. Even with equal weights this approach shows some robustness to missing data, because "uncertain" classifiers tend to contribute equal and therefore small probabilities to each class.…”