Distant speech separation using predicted time–frequency masks from spatial features

Abstract

Speech separation algorithms are faced with a difficult task of producing high degree of separation without containing unwanted artifacts. The time-frequency (T-F) masking technique applies a real-valued (or binary) mask on top of the signal's spectrum to filter out unwanted components. The practical difficulty lies in the mask estimation. Often, using efficient masks engineered for separation performance leads to presence of unwanted musical noise artifacts in the separated signal. This lowers the perceptual quality and intelligibility of the output.Microphone arrays have been long studied for processing of distant speech. This work uses a feed-forward neural network for mapping microphone array's spatial features into a T-F mask. Wiener filter is used as a desired mask for training the neural network using speech examples in simulated setting. The T-F masks predicted by the neural network are combined to obtain an enhanced separation mask that exploits the information regarding interference between all sources. The final mask is applied to the delay-and-sum beamformer (DSB) output.The algorithm's objective separation capability in conjunction with the separated speech intelligibility is tested with recorded speech from distant talkers in two rooms from two distances. The results show improvement in instrumental measure for intelligibility and frequency-weighted SNR over complex-valued non-negative matrix factorization (CNMF) source separation approach, spatial sound source separation, and conventional beamforming methods such as the DSB and minimum variance distortionless response (MVDR).

Multichannel Speech Separation with Recurrent Neural Networks from High-Order Ambisonics Recordings

“…However, when the sources are too close for the beamformer to discriminate (25˚), the mask returned by the network is of no help for the multichannel filter, which performs as badly as the simple beamformer and barely improves compared to the mixture. This can be overcome by feeding the network witĥ n 1 (11). When the speakers are 90˚apart, this does not significantly improve the already good performance of the system.…”

“…a mask [8][9][10]. In the multichannel case, several approaches have been proposed to pass spatial information directly to a DNN, for instance using phase difference features between non-coincident microphones [11] or coherence features [12]. However, in these two studies, the mask estimated by the DNN is still applied as a single-channel filter only.…”

A Phase-Based Time-Frequency Masking for Multi-Channel Speech Enhancement in Domestic Environments

“…In this direction, further improvements could be obtained by removing the gaussianity assumption and employing more articulated modeling of the phase distribution. For instance, following recent trends in multi-channel enhancement, neural networks could be used [23,24,25]. A further limitation of the proposed approach is the anechoic phase modeling in Eq.…”

All-Neural Multi-Channel Speech Enhancement

“…In addition, it cannot be applied when the sources are spatially close to each other. Conventional postfiltering techniques, which are mainly based on signal statistics and conventional single-channel speech enhancement [5], [1] or spatial filters computed using phase information [6], [7], [8], [1], usually cannot achieve high-quality noise reduction in reverberant multi-source environments.…”