Combining Spectral and Spatial Features for Deep Learning Based Blind Speaker Separation

Wang, Zhong-Qiu; Wang, DeLiang

doi:10.1109/taslp.2018.2881912

Cited by 122 publications

(55 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…We simulated a spatialized reverberant dataset derived from Wall Street Journal 0 (WSJ0) 2-mix corpus, which are open and well-studied datasets used for speech separation [9,[16][17][18]. There are 20,000, 5,000 and 3,000 multi-channel, reverberant, two-speaker mixed speech in training, development and test set respectively.…”

Section: Datasetmentioning

confidence: 99%

Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning

Zhang

Chen

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Hand-crafted spatial features (e.g., inter-channel phase difference, IPD) play a fundamental role in recent deep learning based multi-channel speech separation (MCSS) methods. However, these manually designed spatial features are hard to incorporate into the end-to-end optimized MCSS framework. In this work, we propose an integrated architecture for learning spatial features directly from the multi-channel speech waveforms within an end-to-end speech separation framework. In this architecture, time-domain filters spanning signal channels are trained to perform adaptive spatial filtering. These filters are implemented by a 2d convolution (conv2d) layer and their parameters are optimized using a speech separation objective function in a purely data-driven fashion. Furthermore, inspired by the IPD formulation, we design a conv2d kernel to compute the inter-channel convolution differences (ICDs), which are expected to provide the spatial cues that help to distinguish the directional sources. Evaluation results on simulated multi-channel reverberant WSJ0 2-mix dataset demonstrate that our proposed ICD based MCSS model improves the overall signal-to-distortion ratio by 10.4% over the IPD based MCSS model.

show abstract

Section: Datasetmentioning

confidence: 99%

Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning

Zhang

Chen

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Step 1, Estimating the first DOA: In the first step we estimate the DOA of a first speaker using a neural network. The cosines and sines of the phase differences between all pairs of microphones [6,14], called cosine-sine interchannel phase difference (CSIPD) features, and the short-term magnitude spectrum of one of the channels (in the following, channel 1) are used as input features (see Section 3.2):…”

Section: Estimating the Sourcesmentioning

confidence: 99%

“…Computational auditory scene analysis [5] based systems cluster time-frequency bins dominated by the same source using cues such as pitch and interaural time and level differences. In [6], a neural network is trained using the phase differences of the multichannel short-time Fourier transform (STFT) as input features to learn such cues. Deep clustering [1] is then used to associate the cues to the right source.…”

Section: Introductionmentioning

confidence: 99%

SLOGD: Speaker Location Guided Deflation Approach to Speech Separation

Sivasankaran

Fohr

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Speech separation is the process of separating multiple speakers from an audio recording. In this work we propose to separate the sources using a Speaker LOcalization Guided Deflation (SLOGD) approach wherein we estimate the sources iteratively. In each iteration we first estimate the location of the speaker and use it to estimate a mask corresponding to the localized speaker. The estimated source is removed from the mixture before estimating the location and mask of the next source. Experiments are conducted on a reverberated, noisy multichannel version of the well-studied WSJ-2MIX dataset using word error rate (WER) as a metric. The proposed method achieves a WER of 44.2 %, a 34% relative improvement over the system without separation and 17% relative improvement over Conv-TasNet.

show abstract

“…There exist several unsupervised approaches for multichannel speech source separation including independent component analysis based methods [6][7][8] and local Gaussian model (LGM) based method [9]. Meanwhile, motivated by the strong capability of a deep neural network (DNN) to model a spectrogram of a speech, supervised approaches have been paid increasing attention [10][11][12][13].…”

Section: Introductionmentioning

confidence: 99%

“…Recently, using PIT, speaker-independent multi-talker separation by mask-based beamforming was presented [10][11][12]. In these studies, loss functions designed for monaural speech enhancement/separation, such as the phase sensitive approximation (PSA) [27], are employed in the training.…”

Section: Introductionmentioning

confidence: 99%

Multichannel Loss Function for Supervised Speech Source Separation by Mask-Based Beamforming

2019

View full text Add to dashboard Cite

In this paper, we propose two mask-based beamforming methods using a deep neural network (DNN) trained by multichannel loss functions. Beamforming technique using time-frequency (TF)-masks estimated by a DNN have been applied to many applications where TF-masks are used for estimating spatial covariance matrices. To train a DNN for mask-based beamforming, loss functions designed for monaural speech enhancement/separation have been employed. Although such a training criterion is simple, it does not directly correspond to the performance of mask-based beamforming. To overcome this problem, we use multichannel loss functions which evaluate the estimated spatial covariance matrices based on the multichannel Itakura-Saito divergence. DNNs trained by the multichannel loss functions can be applied to construct several beamformers. Experimental results confirmed their effectiveness and robustness to microphone configurations.

show abstract

Combining Spectral and Spatial Features for Deep Learning Based Blind Speaker Separation

Cited by 122 publications

References 41 publications

Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning

Enhancing End-to-End Multi-Channel Speech Separation Via Spatial Feature Learning

SLOGD: Speaker Location Guided Deflation Approach to Speech Separation

Multichannel Loss Function for Supervised Speech Source Separation by Mask-Based Beamforming

Contact Info

Product

Resources

About