DNN-based mask estimation for distributed speech enhancement in spatially unconstrained microphone arrays

Abstract

Deep neural network (DNN)-based speech enhancement algorithms in microphone arrays have now proven to be efficient solutions to speech understanding and speech recognition in noisy environments. However, in the context of adhoc microphone arrays, many challenges remain and raise the need for distributed processing. In this paper, we propose to extend a previously introduced distributed DNN-based timefrequency mask estimation scheme that can efficiently use spatial information in form of so-called compressed signals which are pre-filtered target estimations. We study the performance of this algorithm under realistic acoustic conditions and investigate practical aspects of its optimal application. We show that the nodes in the microphone array cooperate by taking profit of their spatial coverage in the room. We also propose to use the compressed signals not only to convey the target estimation but also the noise estimation in order to exploit the acoustic diversity recorded throughout the microphone array.

Attention-based distributed speech enhancement for unconstrained microphone arrays with varying number of nodes

“…This achieved better performance than with an oracle VAD. In an extended study, we generalized these results to real-life scenarios and showed that sending the noise estimate rather than the target estimate could improve the performance depending on the source to interferences ratio (SIR) at the receiving node [17]. To take full advantage from the spatial coverage of the distributed microphone array, in the following, both the target and noise 1 estimates will be sent as represented in Figure 1.…”

“…It is denoted "SN". The second model is the model that has been used in our previous experiments [17]. It is a multi-node neural network that estimates the mask based on the local reference signal and the compressed signals sent from the other nodes.…”

“…The dataset used to train and test our proposed solution is the same as the one of our previous work [17]. It consists of simulations of typical shoebox-like rooms with one target source and one noise source randomly laid in the room.…”

“…It is split into two non-overlapping subsets of Freesound users for the training and testing sets. Some speech-shaped noise was also used to train the DNN because it was shown to improve the robustness of the DNN [17].…”

“…It processes the information captured over the whole microphone array but limits the number of signals exchanged between nodes and operates in arrays with a varying number of devices. This solution is based on our previous work [17], which benefits from a distributed multichannel Wiener filter (MWF) [1] to alleviate the constraints on the fusion center. It also benefits from the modelling power of DNNs which proved to efficiently use spatial information for a more precise time-frequency (TF) mask estimation.…”