End-to-End Multi-Channel Speech Separation

Gu, Rongzhi; Wu, Jian; Zhang, Shi-Xiong; Chen, Lianwu; Xu, Yong; Yu, Meng; Su, Dan; Zou, Yuexian; Yu, Dong

doi:10.48550/arxiv.1905.06286

Cited by 25 publications

(46 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Because the source code of TSB is not publicly available, we implemented this algorithm in [26] and the reproduced results shown in "TSB (our), TSB-IPD (our)" are slightly better than the ones in [9]. Instead of using IPD to exploit the multi-channel spatial information, the parallel encoder (Para-Enc) [16] was proposed Table 1: SDR/SI-SDR (dB) performance of different target speech extraction systems. "SA" represents the speaker adaptation is also performed on the parallel encoder output.…”

Section: Results In Sdr/si-sdrmentioning

confidence: 99%

“…This decorrelation is performed on each dimension of all the multi-channel encoder representations of input mixtures, it is used to extract the inter-channel differential spatial information to learn difference between individual source signals of input mixture. Results in [15] have already shown that our original CD significantly improved the TSE performance over IPD features, however, the performance gains over the parallel encoder architecture [16] are still limited.…”

Section: Introductionmentioning

confidence: 90%

See 1 more Smart Citation

Improving Channel Decorrelation for Multi-Channel Target Speech Extraction

Han¹,

Rao

Wang

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

Target speech extraction has attracted widespread attention. When microphone arrays are available, the additional spatial information can be helpful in extracting the target speech. We have recently proposed a channel decorrelation (CD) mechanism to extract the inter-channel differential information to enhance the reference channel encoder representation. Although the proposed mechanism has shown promising results for extracting the target speech from mixtures, the extraction performance is still limited by the nature of the original decorrelation theory. In this paper, we propose two methods to broaden the horizon of the original channel decorrelation, by replacing the original softmax-based inter-channel similarity between encoder representations, using an unrolled probability and a normalized cosine-based similarity at the dimensional-level. Moreover, new combination strategies of the CD-based spatial information and target speaker adaptation of parallel encoder outputs are also investigated. Experiments on the reverberant WSJ0 2mix show that the improved CD can result in more discriminative differential information and the new adaptation strategy is also very effective to improve the target speech extraction.

show abstract

Section: Results In Sdr/si-sdrmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 90%

Improving Channel Decorrelation for Multi-Channel Target Speech Extraction

Han¹,

Rao

Wang

et al. 2021

Interspeech 2021

View full text Add to dashboard Cite

show abstract

“…In our model, both STFT and iSTFT are implemented by convolu- Architecture of the proposed HGCN. tion [17]. So, the input to the encoder is the noisy complex spectrum, denoted as S = Cat(S r , S i ) ∈ R T ×2F , where S r and S i represent the real and imaginary parts of the spectrum respectively.…”

Section: Coarse Enhancement Modulementioning

confidence: 99%

HGCN: Harmonic gated compensation network for speech enhancement

Wang¹,

Zhu²,

Gao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Mask processing in the time-frequency (T-F) domain through the neural network has been one of the mainstreams for single-channel speech enhancement. However, it is hard for most models to handle the situation when harmonics are partially masked by noise. To tackle this challenge, we propose a harmonic gated compensation network (HGCN). We design a high-resolution harmonic integral spectrum to improve the accuracy of harmonic locations prediction. Then we add voice activity detection (VAD) and voiced region detection (VRD) to the convolutional recurrent network (CRN) to filter harmonic locations. Finally, the harmonic gating mechanism is used to guide the compensation model to adjust the coarse results from CRN to obtain the refinedly enhanced results. Our experiments show HGCN achieves substantial gain over a number of advanced approaches in the community.

show abstract

“…As is illustrated in Figure 2, three types of audio features including the complex spectrum, the inter-microphone phase differences (IPDs) [25] and location-guided angle feature (AF) [26,27] are adopted as the audio inputs. The complex spectrum of all the microphone array channels are first computed through short-time Fourier transform (STFT).…”

Section: Audio Inputsmentioning

confidence: 99%

Mixed Precision DNN Qunatization for Overlapped Speech Separation and Recognition

Xu¹,

Yu²,

Liu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recognition of overlapped speech has been a highly challenging task to date. State-of-the-art multi-channel speech separation system are becoming increasingly complex and expensive for practical applications. To this end, low-bit neural network quantization provides a powerful solution to dramatically reduce their model size. However, current quantization methods are based on uniform precision and fail to account for the varying performance sensitivity at different model components to quantization errors. In this paper, novel mixed precision DNN quantization methods are proposed by applying locally variable bit-widths to individual TCN components of a TF masking based multi-channel speech separation system. The optimal local precision settings are automatically learned using three techniques. The first two approaches utilize quantization sensitivity metrics based on either the mean square error (MSE) loss function curvature, or the KL-divergence measured between full precision and quantized separation models. The third approach is based on mixed precision neural architecture search. Experiments conducted on the LRS3-TED corpus simulated overlapped speech data suggest that the proposed mixed precision quantization techniques consistently outperform the uniform precision baseline speech separation systems of comparable bit-widths in terms of SI-SNR and PESQ scores as well as word error rate (WER) reductions up to 2.88% absolute (8% relative).

show abstract

End-to-End Multi-Channel Speech Separation

Cited by 25 publications

References 22 publications

Improving Channel Decorrelation for Multi-Channel Target Speech Extraction

Improving Channel Decorrelation for Multi-Channel Target Speech Extraction

HGCN: Harmonic gated compensation network for speech enhancement

Mixed Precision DNN Qunatization for Overlapped Speech Separation and Recognition

Contact Info

Product

Resources

About