Recursive Speech Separation for Unknown Number of Speakers

Abstract

In this paper we propose a method of single-channel speakerindependent multi-speaker speech separation for an unknown number of speakers. As opposed to previous works, in which the number of speakers is assumed to be known in advance and speech separation models are specific for the number of speakers, our proposed method can be applied to cases with different numbers of speakers using a single model by recursively separating a speaker. To make the separation model recursively applicable, we propose one-and-rest permutation invariant training (OR-PIT). Evaluation on WSJ0-2mix and WSJ0-3mix datasets show that our proposed method achieves state-ofthe-art results for two-and three-speaker mixtures with a single model. Moreover, the same model can separate four-speaker mixture, which was never seen during the training. We further propose the detection of the number of speakers in a mixture during recursive separation and show that this approach can more accurately estimate the number of speakers than detection in advance by using a deep neural network based classifier.

Tackling Real Noisy Reverberant Meetings with All-Neural Source Separation, Counting, and Diarization System

“…It sometimes confidently extracts or tracks two different speakers' signals with one speaker embedding vector {z b,i } 1≤b≤B , probably because their voice characteristics are similar from the system's point of view. This type of error should be reduced by, for example, employing more advanced NN architecture [18], and increasing the number of speakers in training data like we propose in [29].…”

Meta-Learning Extractors for Music Source Separation

“…A major design choice in music source separation models is whether to (1) train a separate model for each instrument [12], (2) to use a single class-conditional model, or (3) to use an instrument agnostic approach [16]. Our approach aims to combine the advantages of the first two; the high-precision of independent models, with improved optimization via parameter sharing in single models.…”

SLOGD: Speaker Location Guided Deflation Approach to Speech Separation

“…A few earlier works have proposed to iteratively estimate the sources using deep neural networks in a single-channel setting [2,12]. To the best of our knowledge, this paper is the first such study in a multichannel setting, where we estimate both the DOAs and the masks of all speakers.…”