Multichannel Spatial Clustering Using Model-Based Source Separation

Mandel, Michael I.; Barker, Jon

doi:10.1007/978-3-319-64680-0_3

Cited by 3 publications

(2 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The rationale lies in that, when sources are (W-) disjoint orthogonal, the binaural cues will form clusters within each frequency band for spatially separated directional sources with different time delays. This is also the theoretical basis of spatial clustering technique [65], [66]. These spatial cues have been proven effective in deep learning-based frequency domain separation methods, especially when combined with spectral feature (e.g., logarithm power spectra, LPS) at input level [16], [19], [36], [40], [42].…”

Section: Introductionmentioning

confidence: 98%

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

Gu,

Zou

2020

Preprint

View full text Add to dashboard Cite

Target speech separation refers to extracting the target speaker's speech from mixed signals. Despite the recent advances in deep learning based close-talk speech separation, the applications to real-world are still an open issue. Two main challenges are the complex acoustic environment and the realtime processing requirement. To address these challenges, we propose a temporal-spatial neural filter, which directly estimates the target speech waveform from multi-speaker mixture in reverberant environments, assisted with directional information of the speaker(s). Firstly, against variations brought by complex environment, the key idea is to increase the acoustic representation completeness through the jointly modeling of temporal, spectral and spatial discriminability between the target and interference source. Specifically, temporal, spectral, spatial along with the designed directional features are integrated to create a joint acoustic representation.Secondly, to reduce the latency, we design a fullyconvolutional autoencoder framework, which is purely end-to-end and single-pass. All the feature computation is implemented by the network layers and operations to speed up the separation procedure. Evaluation is conducted on simulated reverberant dataset WSJ0-2mix and WSJ0-3mix under speaker-independent scenario. Experimental results demonstrate that the proposed method outperforms state-of-the-art deep learning based multichannel approaches with fewer parameters and faster processing speed. Furthermore, the proposed temporal-spatial neural filter can handle mixtures with varying and unknown number of speakers and exhibits persistent performance even when existing a direction estimation error. Codes and models will be released soon.

show abstract

Section: Introductionmentioning

confidence: 98%

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

Gu,

Zou

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…When multiple microphones are available, spatial information can be leveraged for better separation, as speaker sources are directional and usually spatially separated in realworld environments. One stream of research to exploit this information is focused on spatial clustering [10], [11], [12], [13], [14], which clusters individual T-F units according to their spatial origins under the speech sparsity assumption [8], [15], using spatial cues such as interchannel time, phase or level differences (ITDs/IPDs/ILDs) and directional statistics. However, these approaches typically only consider spatial information, which is insufficient for separation in reverberant environments or when sound sources are close to one another.…”

Section: Introductionmentioning

confidence: 99%

Integrating Spectral and Spatial Features for Multi-Channel Speaker Separation

Wang

2018

Interspeech 2018

View full text Add to dashboard Cite

This paper tightly integrates spectral and spatial information for deep learning based multi-channel speaker separation. The key idea is to localize individual speakers so that an enhancement network can be used to separate the speaker from an estimated direction and with specific spectral characteristics. To determine the direction of the speaker of interest, we identify time-frequency (T-F) units dominated by that speaker and only use them for direction of arrival (DOA) estimation. The speaker dominance at each T-F unit is determined by a twochannel permutation invariant training network, which combines spectral and interchannel phase patterns at the input feature level. In addition, beamforming is tightly integrated in the proposed system by exploiting the magnitudes and phase produced by T-F masking based beamforming. Strong separation performance has been observed on a spatialized reverberant version of the wsj0-2mix corpus.

show abstract

Combining Spectral and Spatial Features for Deep Learning Based Blind Speaker Separation

Wang

2019

IEEE/ACM Trans. Audio Speech Lang. Process.

122

View full text Add to dashboard Cite

Multichannel Spatial Clustering Using Model-Based Source Separation

Cited by 3 publications

References 47 publications

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

Integrating Spectral and Spatial Features for Multi-Channel Speaker Separation

Combining Spectral and Spatial Features for Deep Learning Based Blind Speaker Separation

Contact Info

Product

Resources

About