TPARN: Triple-Path Attentive Recurrent Network for Time-Domain Multichannel Speech Enhancement

Pandey, Ashutosh; Xu, Buye; Kumar, Anurag; Donley, Jacob; Calamia, Paul; Wang, DeLiang

doi:10.1109/icassp43922.2022.9747373

Cited by 21 publications

(14 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We demonstrate the improved performance, low data bias, and environment bias of the proposed model through various simulated datasets. Performance comparison with state-ofthe-art models on three datasets (spatialized WSJCAM0 [8], spatialized DNS challenge [5], and L3DAS22 [52]) confirms that our proposed model has lower computational complexity and higher performance enhancement. To further investigate the real-world applicability and scalability of our model, we conduct experiments on real noisy and reverberant speech recorded in an office environment.…”

Section: Introductionmentioning

confidence: 54%

“…The primary objective of multichannel speech enhancement is to restore clean speech by reducing noise and reverberation from measured multichannel speech. While it is possible to design multi-input/multioutput models [4], [5] for restoring multichannel clean speech, the majority of approaches focus on multi-input/single-output models [6]- [15] that aim to restore the clean speech of a reference channel.…”

Section: Introductionmentioning

confidence: 99%

“…Since SepFormer, several state-of-the-art speech enhancement and separation models have utilized transformers in network design [15], [34], [36], [36], [45]. For multichannel speech enhancement, a triple-path attentive recurrent network (TPARN) [5] utilizes spatial, long-term, and short-term temporal relations using an attentive recurrent network (ARN). Pandey et al [11] arXiv:2308.15777v1 [eess.AS] 30 Aug 2023 introduced an attentive dense convolutional network (ADCN) that utilizes attention for capturing time-domain relationships.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

DeFT-AN: Dense Frequency-Time Attentive Network for Multichannel Speech Enhancement

Lee

Choi

2023

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

In this work, we present DeFTAN-II, an efficient multichannel speech enhancement model based on transformer architecture and subgroup processing. Despite the success of transformers in speech enhancement, they face challenges in capturing local relations, reducing the high computational complexity, and lowering memory usage. To address these limitations, we introduce subgroup processing in our model, combining subgroups of locally emphasized features with other subgroups containing original features. The subgroup processing is implemented in several blocks of the proposed network. In the proposed split dense blocks extracting spatial features, a pair of subgroups is sequentially concatenated and processed by convolution layers to effectively reduce the computational complexity and memory usage. For the F-and T-transformers extracting temporal and spectral relations, we introduce crossattention between subgroups to identify relationships between locally emphasized and non-emphasized features. The dualpath feedforward network then aggregates attended features in terms of the gating of local features processed by dilated convolutions. Through extensive comparisons with state-of-theart multichannel speech enhancement models, we demonstrate that DeFTAN-II with subgroup processing outperforms existing methods at significantly lower computational complexity. Moreover, we evaluate the model's generalization capability on realworld data without fine-tuning, which further demonstrates its effectiveness in practical scenarios.

show abstract

Section: Introductionmentioning

confidence: 54%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

DeFT-AN: Dense Frequency-Time Attentive Network for Multichannel Speech Enhancement

Lee

Choi

2023

IEEE Signal Process. Lett.

View full text Add to dashboard Cite

show abstract

“…The inter-channel is also bypassed for recordings with J = 1. The inter-channel transformer is similar to the strategy implemented in [23] for multi-channel speech enhancement.…”

Section: B Masking Networkmentioning

confidence: 99%

Speaker Counting and Separation From Single-Channel Noisy Mixtures

Chetupalli

Habets

2023

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

We address the problem of speaker counting and separation from a noisy, single-channel, multi-source, recording. Most of the works in the literature assume mixtures containing two to five speakers. In this work, we consider noisy speech mixtures with one to five speakers and noise-only recordings. We propose a deep neural network (DNN) architecture, that predicts a speaker count of zero for noise-only recordings and predicts the individual clean speaker signals and speaker count for mixtures of one to five speakers. The DNN is composed of transformer layers and processes the recordings using the long-time and short-time sequence modeling approach to masking in a learned time-feature domain. The network uses an encoder-decoder attractor module with long-short term memory units to generate a variable number of outputs. The network is trained with simulated noisy speech mixtures composed of the speech recordings from WSJ0 corpus, and noise recordings from the WHAM! corpus. We show that the network achieves 99% speaker counting accuracy and more than 19 dB improvement in the scale-invariant signal-to-noise ratio for mixtures of up to three speakers.

show abstract

“…However, the channel selection approach is not optimized. Recently, many works explored advanced channel selection approaches for speech enhancement [24], [25], speech separation [26]- [28], speech recognition [29], and speaker recognition [30], [31]. Particularly, Liang et al [30] and Cai et al [31] independently proposed end-to-end speaker verification with ad-hoc microphone arrays, where an inter-channel attention-based channel reweighting method was developed to fuse utterance-level speaker features from all channels.…”

Section: Arxiv:230701386v1 [Cssd] 3 Jul 2023mentioning

confidence: 99%

Scaling Sparsemax Based Channel Selection for Speech Recognition with ad-hoc Microphone Arrays

Chen¹,

Zhang²

2021

Interspeech 2021

View full text Add to dashboard Cite

Recently, speech recognition with ad-hoc microphone arrays has received much attention. It is known that channel selection is an important problem of ad-hoc microphone arrays, however, this topic seems far from explored in speech recognition yet, particularly with a large-scale ad-hoc microphone array. To address this problem, we propose a Scaling Sparsemax algorithm for the channel selection problem of the speech recognition with large-scale ad-hoc microphone arrays. Specifically, we first replace the conventional Softmax operator in the stream attention mechanism of a multichannel end-to-end speech recognition system with Sparsemax, which conducts channel selection by forcing the channel weights of noisy channels to zero. Because Sparsemax punishes the weights of many channels to zero harshly, we propose Scaling Sparsemax which punishes the channels mildly by setting the weights of very noisy channels to zero only. Experimental results with ad-hoc microphone arrays of over 30 channels under the conformer speech recognition architecture show that the proposed Scaling Sparsemax yields a word error rate of over 30% lower than Softmax on simulation data sets, and over 20% lower on semi-real data sets, in test scenarios with both matched and mismatched channel numbers.

show abstract

TPARN: Triple-Path Attentive Recurrent Network for Time-Domain Multichannel Speech Enhancement

Cited by 21 publications

References 24 publications

DeFT-AN: Dense Frequency-Time Attentive Network for Multichannel Speech Enhancement

DeFT-AN: Dense Frequency-Time Attentive Network for Multichannel Speech Enhancement

Speaker Counting and Separation From Single-Channel Noisy Mixtures

Scaling Sparsemax Based Channel Selection for Speech Recognition with ad-hoc Microphone Arrays

Contact Info

Product

Resources

About