End-to-End Monaural Speech Separation with Multi-Scale Dynamic Weighted Gated Dilated Convolutional Pyramid Network

Shi, Ziqiang; Lin, Huibin; Liu, Liu; Liu, Rujie; Hayakawa, Shoji; Harada, Shouji; Han, Jiqing

doi:10.21437/interspeech.2019-1292

Cited by 17 publications

(14 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By default, we use the scale-invariant signal-to-noise ratio (SI-SNR) [26] with permutation-invariant training [1] as our objective function. SI-SNR is a widely used objective function for end-to-end speech source separation [8,10,11,15]. In certain experiments (see Section 4.2), in an effort to constrain the scale of the predicted sources from the deep encoder/decoder, we augment the objective function with a power-law term that encourages the model to predict spectra that are of similar magnitude to the ground truth.…”

Section: Objective Functionsmentioning

confidence: 99%

“…On the other hand, Yang et al [15] is another recent work that enhances the separator by utilizing an embedding network and clustering. Their separator also takes advantage of STFT features, whereas our model learns only from [15] 16.9 10M FurcaPy [11] 18.4 N/A waveforms. When comparing their results with ours, we note that by simply increasing the depth of the encoder/decoder one can achieve a similar improvement to Yang et al [15].…”

Section: Deep Encoder/decoder Vs Enhanced Separatorsmentioning

confidence: 99%

“…However, most focus has been devoted to its separator. For example, in [10,11] a parallel and multi-scale separator is proposed, Work done during Berkan Kadıoglu's internship at Dolby Laboratories. Berkan, Michael and Xiaoyu equally contributed to this work.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

An Empirical Study of Conv-Tasnet

Kadıoğlu

Horgan

Liu

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Conv-TasNet is a recently proposed waveform-based deep neural network that achieves state-of-the-art performance in speech source separation. Its architecture consists of a learnable encoder/decoder and a separator that operates on top of this learned space. Various improvements have been proposed to Conv-TasNet. However, they mostly focus on the separator, leaving its encoder/decoder as a (shallow) linear operator. In this paper, we conduct an empirical study of Conv-TasNet and propose an enhancement to the encoder/decoder that is based on a (deep) non-linear variant of it. In addition, we experiment with the larger and more diverse LibriTTS dataset and investigate the generalization capabilities of the studied models when trained on a much larger dataset. We propose cross-dataset evaluation that includes assessing separations from the WSJ0-2mix, Lib-riTTS and VCTK databases. Our results show that enhancements to the encoder/decoder can improve average SI-SNR performance by more than 1 dB. Furthermore, we offer insights into the generalization capabilities of Conv-TasNet and the potential value of improvements to the encoder/decoder.

show abstract

Section: Objective Functionsmentioning

confidence: 99%

Section: Deep Encoder/decoder Vs Enhanced Separatorsmentioning

confidence: 99%

See 1 more Smart Citation

An Empirical Study of Conv-Tasnet

Kadıoğlu

Horgan

Liu

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…This strategy imposes an upper limit on the separation performance. To overcome this problem, time-domain approach is proposed in paper [14] , which directly model the mixture waveform using an encodedecoder framework and has made great progress in recent years [15,16,17,18,19,20,21].…”

Section: Introductionmentioning

confidence: 99%

Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation

2020

View full text Add to dashboard Cite

The dominant speech separation models are based on complex recurrent or convolution neural network that model speech sequences indirectly conditioning on context, such as passing information through many intermediate states in recurrent neural network, leading to suboptimal separation performance. In this paper, we propose a dual-path transformer network (DPT-Net) for end-to-end speech separation, which introduces direct context-awareness in the modeling for speech sequences. By introduces a improved transformer, elements in speech sequences can interact directly, which enables DPTNet can model for the speech sequences with direct context-awareness. The improved transformer in our approach learns the order information of the speech sequences without positional encodings by incorporating a recurrent neural network into the original transformer. In addition, the structure of dual paths makes our model efficient for extremely long speech sequence modeling. Extensive experiments on benchmark datasets show that our approach outperforms the current state-of-the-arts (20.6 dB SDR on the public WSj0-2mix data corpus).

show abstract

“…These structural limitations are circumvented by end-to-end speech separation systems such as TasNet [6], Conv-TasNet [7] or FurcaNext [8,9]. These systems introduce several changes to the STFT magnitude based approaches: First, the training loss is defined in the time domain instead of the STFT domain.…”

Section: Introductionmentioning

confidence: 99%

A Multi-Phase Gammatone Filterbank for Speech Separation Via Tasnet

Ditter

Gerkmann

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this work, we investigate if the learned encoder of the end-to-end convolutional time domain audio separation network (Conv-TasNet) is the key to its recent success, or if the encoder can just as well be replaced by a deterministic hand-crafted filterbank. Motivated by the resemblance of the trained encoder of Conv-TasNet to auditory filterbanks, we propose to employ a deterministic gammatone filterbank. In contrast to a common gammatone filterbank, our filters are restricted to 2 ms length to allow for low-latency processing. Inspired by the encoder learned by Conv-TasNet, in addition to the logarithmically spaced filters, the proposed filterbank holds multiple gammatone filters at the same center frequency with varying phase shifts. We show that replacing the learned encoder with our proposed multi-phase gammatone filterbank (MP-GTF) even leads to a scale-invariant source-to-noise ratio (SI-SNR) improvement of 0.8 dB. Furthermore, in contrast to using the learned encoder we show that the number of filters can be reduced from 512 to 128 without loss of performance.

show abstract

End-to-End Monaural Speech Separation with Multi-Scale Dynamic Weighted Gated Dilated Convolutional Pyramid Network

Cited by 17 publications

References 13 publications

An Empirical Study of Conv-Tasnet

An Empirical Study of Conv-Tasnet

Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation

A Multi-Phase Gammatone Filterbank for Speech Separation Via Tasnet

Contact Info

Product

Resources

About