Improved Speech Separation with Time-and-Frequency Cross-domain Joint Embedding and Clustering

Yang, Gene-Ping; Tuan, Chao-I; Lee, Hung-yi; Lee, Lin-Shan

doi:10.48550/arxiv.1904.07845

Cited by 4 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…a is the projection of b onto a. SI-SNR has also been used in many end-to-end separation models [6,10,30,2].…”

Section: Permutation Invariant Trainingmentioning

confidence: 99%

Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks

Lam

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

Recent research on the time-domain audio separation networks (Tas-Nets) has brought great success to speech separation. Nevertheless, conventional TasNets struggle to satisfy the memory and latency constraints in industrial applications. In this regard, we design a low-cost high-performance architecture, namely, globally attentive locally recurrent (GALR) network. Alike the dual-path RNN (DPRNN), we first split a feature sequence into 2D segments and then process the sequence along both the intra-and inter-segment dimensions. Our main innovation lies in that, on top of features recurrently processed along the inter-segment dimensions, GALR applies a self-attention mechanism to the sequence along the inter-segment dimension, which aggregates context-aware information and also enables parallelization. Our experiments suggest that GALR is a notably more effective network than the prior work. On one hand, with only 1.5M parameters, it has achieved comparable separation performance at a much lower cost with 36.1% less runtime memory and 49.4% fewer computational operations, relative to the DPRNN. On the other hand, in a comparable model size with DPRNN, GALR has consistently outperformed DPRNN in three datasets, in particular, with a substantial margin of 2.4dB absolute improvement of SI-SNRi in the benchmark WSJ0-2mix task.

show abstract

“…a is the projection of b onto a. SI-SNR has also been used in many end-to-end separation models [6,10,30,2].…”

Section: Permutation Invariant Trainingmentioning

confidence: 99%

Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks

Lam

Wang

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…1. For example, we apply the recently proposed scale-invariant signal-to-noise ratio (SI-SNR) [8,18] with PIT [6,7]:…”

Section: Proposed Approachmentioning

confidence: 99%

Mixup-breakdown: A Consistency Training Method for Improving Generalization of Speech Separation Models

Lam

Wang

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Deep-learning based speech separation models confront poor generalization problem that even the state-of-the-art models could abruptly fail when evaluating them in mismatch conditions. To address this problem, we propose an easy-to-implement yet effective consistency based semi-supervised learning (SSL) approach, namely Mixup-Breakdown training (MBT). It learns a teacher model to "breakdown" unlabeled inputs, and the estimated separations are interpolated to produce more useful pseudo "mixup" input-output pairs, on which the consistency regularization could apply for learning a student model. In our experiment, we evaluate MBT under various conditions with ascending degrees of mismatch, including unseen interfering speech, noise, and music, and compare MBT's generalization capability against state-of-the-art supervised learning and SSL approaches. The result indicates that MBT significantly outperforms several strong baselines with up to 13.77% relative SI-SNRi improvement. Moreover, MBT only adds negligible computational overhead to standard training schemes.

show abstract

“…The time-frequency domain method is to establish the frequency domain signal characteristics through short-time Fourier transform, use the T-F [5] method to separate the signal, and finally reconstruct the source waveform through inverse. The time-frequency domain methods mainly include [5,6] . The endto-end time-domain method mainly uses the waveform mixed in the original stage.…”

Section: Introductionmentioning

confidence: 99%

Separation Structure Speech Separation Model Fusing Conformer and NBC

Zhang

Cheng

Ding

2022

J. Phys.: Conf. Ser.

View full text Add to dashboard Cite

The quality of speech separation affects the entire speech technology ecosystem. Aiming at the problems of low utilization of local feature information, insufficient convergence speed, too many calculation parameters and too long calculation time in blind source separation in view of Transformer dual-path cyclic neural network, a blind source separation model based on fusion of Conformer and NBC (Narrow-band Conformer NBC) is proposed. First, for the problems of low utilization of speech local feature information and insufficient convergence speed, the Transformer block is replaced by the Conformer in the block of the dual-path recurrent network. It can improve the utilization and convergence speed of local features. Secondly, the NBC block is used to replace the Transformer block in the inter-block loop. The NBC block simplifies the calculation of vector similarity and vector aggregation, reduces lots of parameters and calculation cost, and reduces the computational complexity of the model. In the experiments on WSJ0-2mix[7] dataset and WHAM[13] dataset, contrast with other models, the convergence speech is faster and the blind source separation effect is better.

show abstract

Improved Speech Separation with Time-and-Frequency Cross-domain Joint Embedding and Clustering

Cited by 4 publications

References 17 publications

Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks

Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks

Mixup-breakdown: A Consistency Training Method for Improving Generalization of Speech Separation Models

Separation Structure Speech Separation Model Fusing Conformer and NBC

Contact Info

Product

Resources

About