Deep Multi-Frame MVDR Filtering for Single-Microphone Speech Enhancement

Abstract

Multi-frame algorithms for single-microphone speech enhancement, e.g., the multi-frame minimum variance distortionless response (MFMVDR) filter, are able to exploit speech correlation across adjacent time frames in the short-time Fourier transform (STFT) domain. Provided that accurate estimates of the required speech interframe correlation vector and the noise correlation matrix are available, it has been shown that the MFMVDR filter yields a substantial noise reduction while hardly introducing any speech distortion. Aiming at merging the speech enhancement potential of the MFMVDR filter and the estimation capability of temporal convolutional networks (TCNs), in this paper we propose to embed the MFMVDR filter within a deep learning framework. The TCNs are trained to map the noisy speech STFT coefficients to the required quantities by minimizing the scale-invariant signal-to-distortion ratio loss function at the MFMVDR filter output. Experimental results show that the proposed deep MFMVDR filter achieves a competitive speech enhancement performance on the Deep Noise Suppression Challenge dataset. In particular, the results show that estimating the parameters of an MFMVDR filter yields a higher performance in terms of PESQ and STOI than directly estimating the multi-frame filter or single-frame masks and than Conv-TasNet.

Dual-Channel Speech Enhancement Using Neural Network Adaptive Beamforming

“…The STOI and PESQ of the noisy and enhanced speeches at -5 dB, 0 dB, and 5 dB are measured , respectively. As shown in Table 1, compared with the MMSE-based approch [18], dual-microphone DNN speech enhancement [19], CRN model [20] and MFMVDR model [21] the proposed method significantly improve the performance. For example, at SNR = 0 dB, the NABDTN method increased STOI by 12.7% and PESQ by 1.08, whereas the MFMVDR only improved STOI by 10.42% and PESQ by 1.02.…”

Sixty Years of Frequency-Domain Monaural Speech Enhancement: From Traditional to Deep Learning Methods

“…It is still challenging to reduce the delay to below 4 ms without affecting performance, although some researchers have tried to solve this problem (Vary, 2006; Schröter et al, 2022; Zheng et al, 2022). Tammen & Doclo (2021) proposed a deep multiframe approach to reduce the delay for hearing aids, and this approach was further extended for binaural noise reduction (Tammen & Doclo, 2022). From the application perspective, future work should concentrate on reducing the complexity, storage, and latency of deep-learning methods to facilitate their application in hearing aids and cochlear implants.…”

MetricGAN+/-: Increasing Robustness of Noise Reduction on Unseen Data

“…Firstly D is trained using (5); as a result the replay buffer now contains both enhanced and de-enhanced data, effectively doubling its size. After D's training, N is trained using (6). Then G is trained as usual.…”