Normalization on the modulation spectrum of the subband temporal envelopes for automatic speech recognition in reverberant environments

Abstract

In this study, we proposed a feature extraction method based on the subband temporal envelopes (STEs) and their normalization for reverberated speech recognition. The STEs were extracted by using a series of constant bandwidth bandpass filters with Hilbert transform followed by a low-pass filtering. In the normalization, both the modulation spectrum (MS) of the subband temporal envelopes of the clean and reverberated speech are normalized to a reference MS calculated from a clean speech data set. Based on the normalized subband MS, the inverse Fourier transform was used to restore the subband temporal envelopes. We tested the proposed method on speech recognition in a reverberant room with different speaker to microphone distance (SMD). For comparison, the recognition performance of using the traditional Mel-cepstral coefficients with mean and variance normalization were used as the baseline. Experimental results showed that, by averaging the SMDs from 50 cm to 400 cm, there was a 44.96% relative improvement by only using subband temporal envelope processing, and further a 15.68% relative improvement by using the normalization on the subband modulation spectrum. Totally, there was about a 53.59% relative improvement, which was better than those of using other temporal filtering and normalization methods.

Features Based on Auditory Physiology and Perception

“…8.2.4, was developed to emphasize the critical temporal modulations (and in so doing emphasizes transitions, roughly models forward masking, and reduces sensitivity to irrelevant steady state convolutional factors). More recently, temporal modulation in subbands was normalized to improve ASR in reverberant environments [60].…”