Separating Varying Numbers of Sources with Auxiliary Autoencoding Loss

Abstract

Many recent source separation systems are designed to separate a fixed number of sources out of a mixture. In the cases where the source activation patterns are unknown, such systems have to either adjust the number of outputs or to identify invalid outputs from the valid ones. Iterative separation methods have gain much attention in the community as they can flexibly decide the number of outputs, however (1) they typically rely on long-term information to determine the stopping time for the iterations, which makes them hard to operate in a causal setting;(2) they lack a "fault tolerance" mechanism when the estimated number of sources is different from the actual number. In this paper, we propose a simple training method, the auxiliary autoencoding permutation invariant training (A2PIT), to alleviate the two issues. A2PIT assumes a fixed number of outputs and uses auxiliary autoencoding losses to force the invalid outputs to be the copies of the input mixture, and detects invalid outputs in a fully unsupervised way during inference phase. Experiment results show that A2PIT is able to improve the separation performance across various numbers of speakers and effectively detect the number of speakers in a mixture.

What’s all the Fuss about Free Universal Sound Separation Data?

“…One drawback of this approach compared to ours is that the sourcerecurrent network needs to be run N times to separate N sources, while our proposed network only needs to run once. More recently, Luo and Mesgarani [9] proposed a separation model trained to predict the mixture for inactive output sources. In contrast, our approach trains the separation model to predict zero for inactive sources, making it easier to determine when sources are inactive.…”

Distributed Speech Separation in Spatially Unconstrained Microphone Arrays

“…This could be addressed within our spatially distributed framework by adapting a dedicated strategy, e.g. in estimating the sources iteratively [21] or by adapting the loss function [22,23].…”

Multi-ACCDOA: Localizing and Detecting Overlapping Sounds from the Same Class with Auxiliary Duplicating Permutation Invariant Training

“…A2PIT is used in the source separation task with a variable number of sources [18]. With the scheme, the mixture signal itself is used as the auxiliary targets instead of low-energy random noise signals when there are fewer targets than tracks [18]. That enables the model to use any objective functions such as scale-invariant signalto-distortion ratio (SI-SDR).…”

“…The idea of replacing auxiliary targets inspires us to use another auxiliary target instead of a zero vector in the SELD task. A similarity threshold between the mixture and the outputs was used during inference to determine valid outputs [18].…”

“…To overcome the overlap problem while maintaining the advantages of the class-wise format, we extended the single-ACCDOA format to a multi-ACCDOA format (a class-and track-wise output format). We also proposed auxiliary duplicating permutation invariant training (ADPIT), inspired by auxiliary autoencoding permutation invariant training (A2PIT) on a source separation task with a variable number of sources [18]. The class-wise ADPIT scheme enables us to solve the track permutation problem with duplicated AC-CDOA vectors, which can help the model maintain the performance in the cases with no overlaps from the same class.…”