Probabilistic Permutation Invariant Training for Speech Separation

Abstract

Single-microphone, speaker-independent speech separation is normally performed through two steps: (i) separating the specific speech sources, and (ii) determining the best output-label assignment to find the separation error. The second step is the main obstacle in training neural networks for speech separation. Recently proposed Permutation Invariant Training (PIT) addresses this problem by determining the output-label assignment which minimizes the separation error. In this study, we show that a major drawback of this technique is the overconfident choice of the output-label assignment, especially in the initial steps of training when the network generates unreliable outputs. To solve this problem, we propose Probabilistic PIT (Prob-PIT) which considers the output-label permutation as a discrete latent random variable with a uniform prior distribution. Prob-PIT defines a log-likelihood function based on the prior distributions and the separation errors of all permutations; it trains the speech separation networks by maximizing the loglikelihood function. Prob-PIT can be easily implemented by replacing the minimum function of PIT with a soft-minimum function. We evaluate our approach for speech separation on both TIMIT and CHiME datasets. The results show that the proposed method significantly outperforms PIT in terms of Signal to Distortion Ratio and Signal to Interference Ratio.

Interrupted and Cascaded Permutation Invariant Training for Speech Separation

“…Row (a) is for the baseline separation model TasNet-v2 [4] we used throughout this work. Rows (b)(c) are respectively the latest version of TasNet and our implementation using Prob-PIT [14] with TasNet-v2. Row (d) is for our previous work of cross-domain joint clustering which was not used here at all.…”

On Permutation Invariant Training For Speech Source Separation

“…Prob-PIT [7] considers the probabilities of all utterance level permutations, rather than just the best one, improving the initial training stage when wrong alignments are likely to happen. A similar idea is employed by Yang et al [8], who trained a Conv-TasNet with uPIT and fixed alignments in turns, reporting 17.5 dB SI-SNRi.…”

“…However, the permutation frequently changes over frames at inference time. Improvements to PIT roughly fall into two categories: (i) designing a permutation (or speaker) tracking algorithm for tPIT [2,4,5]; and (ii) designing better uPIT objectives to further strengthen permutation consistency [6][7][8][9]. Along these two lines, our work takes a close look at tPIT+clustering, a recent idea introduced by Deep CASA [2], that targets at accurate frame level separation (tPIT) and speaker tracking (clustering) in two stages.…”

Frame-Based Overlapping Speech Detection Using Convolutional Neural Networks

“…In this work we use a multi-speaker, sentence-based corpus called GRID, which has been used in monaural speech separation and recognition challenge [18]. Also, this dataset has been used in several studies [12,19] for overlapping speech detection and separation. This corpus contains 34 speakers, 16 female and 18 male speakers, each narrating 1000 sentence.…”