A Hybrid Approach for Speech Enhancement Using MoG Model and Neural Network Phoneme Classifier

Abstract

Abstract-In this paper we present a single-microphone speech enhancement algorithm. A hybrid approach is proposed merging the generative Mixture of Gaussians (MoG) model and the discriminative neural network (NN). The proposed algorithm is executed in two phases, the training phase, which does not recur, and the test phase. First, the noise-free speech power spectral density (PSD) is modeled as a MoG, representing the phoneme based diversity in the speech signal. An NN is then trained with phoneme labeled database for phoneme classification with mel-frequency cepstral coefficients (MFCC) as the input features. Given the phoneme classification results, an speech presence probability (SPP) is obtained using both the generative and discriminative models. Soft spectral subtraction is then executed while simultaneously, the noise estimation is updated. The discriminative NN maintain the continuity of the speech and the generative phoneme-based MoG preserves the speech spectral structure. Extensive experimental study using real speech and noise signals is provided. We also compare the proposed algorithm with alternative speech enhancement algorithms. We show that we obtain a significant improvement over previous methods in terms of both speech quality measures and speech recognition results.

Speaker extraction using LCMV beamformer with DNN-based SPP and RTF identification scheme

“…The noise covariance matrix is initialized by averaging the first frames of the utterance, assumed to be noise-only frames. The NN-MM algorithm [9] is then applied to the reference microphone to extract an SPP map. A VAD is calculated based on the SPP detector and used to control the noise estimation update.…”

“…The classification results of the SPI control the RTFsmatrix update. Finally, the LCMV is applied to the noisy input, followed by a postfilter based on the NN-MM algorithm [9].…”

“…We apply the NN-MM algorithm [9] to the BF output to enhance the noisy signal. The NN-MM algorithm utilizes a phoneme-based Mixture of Gaussians (MoG) where each Gaussian represents a different phoneme, and a trained deep neural network (DNN) phonemeclassifier, which classifies time-frames to one of the phonemes in the phoneme-based MoG.…”

“…By merging the generative MoG and the discriminative DNN, the NN-MM constructs a timefrequency SPP map ρ(l, k). A soft spectral attenuation, which was found useful for speech enhancement [9], [11], is then applied to the BF output:…”

“…In this paper, we present a practical end-to-end implementation of a multichannel speech enhancement system based on the LCMV beamformer and a post-processing stage. A recently proposed neural network mixture-maximum (NN-MM) algorithm [9] is utilized to derive a voice activity detector (VAD). A new speaker position identifier (SPI) is proposed based on a pre-trained RTF library.…”