End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation

IEEE/ACM Trans. Audio Speech Lang. Process.

Mesgarani

2021

Self Cite

Despite the recent progress on neural network architectures for speech separation, the balance between the model size, model complexity and model performance is still an important and challenging problem for the deployment of such models to low-resource platforms. In this paper, we propose two simple modules, group communication and context codec, that can be easily applied to a wide range of architectures to jointly decrease the model size and complexity without sacrificing the performance. A group communication module splits a highdimensional feature into groups of low-dimensional features and captures the inter-group dependency. A separation module with a significantly smaller model size can then be shared by all the groups. A context codec module, containing a context encoder and a context decoder, is designed as a learnable downsampling and upsampling module to decrease the length of a sequential feature processed by the separation module. The combination of the group communication and the context codec modules is referred to as the GC3 design. Experimental results show that applying GC3 on multiple network architectures for speech separation can achieve on-par or better performance with as small as 2.5% model size and 17.6% model complexity, respectively.

Section: Experiments Configurations a Data Simulationmentioning

confidence: 99%

Section: B Model Configurations 1) Separation Pipeline Configurationsmentioning

confidence: 99%

Group Communication With Context Codec for Lightweight Source Separation

IEEE/ACM Trans. Audio Speech Lang. Process.

Mesgarani

2021

Self Cite

“…We evaluate our approach on a simulated noisy reverberant twospeaker dataset [31]. 20000, 5000 and 3000 4-second long utterances are simulated for training, validation and test sets, respectively.…”

Section: Datasetmentioning

confidence: 99%

Ultra-Lightweight Speech Separation Via Group Communication

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Mesgarani

2021

Self Cite

Model size and complexity remain the biggest challenges in the deployment of speech enhancement and separation systems on lowresource devices such as earphones and hearing aids. Although methods such as compression, distillation and quantization can be applied to large models, they often come with a cost on the model performance. In this paper, we provide a simple model design paradigm that explicitly designs ultra-lightweight models without sacrificing the performance. Motivated by the sub-band frequency-LSTM (F-LSTM) architectures, we introduce the group communication (GroupComm), where a feature vector is split into smaller groups and a small processing block is used to perform intergroup communication. Unlike standard F-LSTM models where the sub-band outputs are concatenated, an ultra-small module is applied on all the groups in parallel, which allows a significant decrease on the model size. Experiment results show that comparing with a strong baseline model which is already lightweight, GroupComm can achieve on par performance with 35.6 times fewer parameters and 2.3 times fewer operations.

“…We evaluate the different model configurations on a simulated noisy reverberant two-speaker dataset [27]. 20000, 5000 and 3000 4second long utterances are simulated for training, validation and test sets, respectively.…”

Section: Datasetmentioning

confidence: 99%

Rethinking The Separation Layers In Speech Separation Networks

Chen

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

et al. 2021

Self Cite

Modules in all existing speech separation networks can be categorized into single-input-multi-output (SIMO) modules and singleinput-single-output (SISO) modules. SIMO modules generate more outputs than input, and SISO modules keep the numbers of input and output the same. While the majority of separation models only contain SIMO architectures, it has also been shown that certain two-stage separation systems integrated with a post-enhancement SISO module can improve the separation quality. Why performance improvements can be achieved by incorporating the SISO modules? Are SIMO modules always necessary? In this paper, we empirically examine those questions by designing models with varying configurations in the SIMO and SISO modules. We show that comparing with the standard SIMO-only design, a mixed SIMO-SISO design with a same model size is able to improve the separation performance especially under low-overlap conditions. We further validate the necessity of SIMO modules and show that SISO-only models are still able to perform separation without sacrificing the performance. The observations allow us to rethink the model design paradigm and present different views on how the separation is performed.