A Nested U-Net With Self-Attention and Dense Connectivity for Monaural Speech Enhancement

Xiang, Xiaoxiao; Zhang, Xiaojuan; Chen, Haozhe

doi:10.1109/lsp.2021.3128374

Cited by 41 publications

(6 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The model performance evaluated in terms of perceptual evaluation of speech quality (PESQ) [35] and short-term objective intelligibility (STOI) [36]. The performance of the TFANUNet model is compared against following baselines: CRN [11] , TCNN [37], DCCRN [38], CS-CRN [39], DeepxiMMSE [40], MASENet [25], SADNUNet [17], and MAB-CED [26].…”

Section: Experimental Results Analysismentioning

confidence: 99%

“…Using such dense connectivity allows information to flow at maximum speed and be in-depth with maximum depth, while minimising the complexity of the model by effectively reusing intermediate representations between layers. In [16][17][18][19][20], the authors proposed a DenseNet to reduce the number of extended dilated convolutional layers and cover the large receptive area. In DenseNet, the features of the early and later layers are directly merged into a single convolutional layer via dense skip connectivity.…”

Section: Introductionmentioning

confidence: 99%

“…The outputs of the MASENet encoder blocks are recalibrated by the attention block and highlight informative details. In the SADNUNet [17]model, the encoder and decoder model uses nested UNet and dense blocks to extract local and contextual features from speech. All outputs from the encoder group are recalibrated by the selfattention block (SA), highlighting informative details and reducing unwanted features.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Prathipati,

Chakravarthy

2024

Eng. Res. Express

View full text Add to dashboard Cite

Deep-learning models have used attention mechanisms to improve the quality and intelligibility of noisy speech, demonstrating the effectiveness of attention mechanisms. We rely on either spatial or temporal-based attention mechanisms, resulting in severe information loss. In this paper, a time-frequency attention mechanism with a nested U-network (TFANUNet) is proposed for single-channel speech enhancement. By using time-frequency attention (TFA), learns the channel, frequency and time information which is more significant for speech enhancement. Basically, the proposed model is an encoder-decoder model, where each layer in the encoder and decoder is followed by a nested dense residual dilated DensNet (NDRD) based multi-scale context aggression block. NDRD involves multiple dilated convolution with different dilatation factors to explore the large receptive area at different scales simultaneously. NDRD avoids the aliasing problem in DenseNet. We integrated the TFA and NDRD blocks into the proposed model to enable refined feature set extraction without information loss and utterance-level context aggregation, respectively. The proposed TFANUNet model results outperform baselines in terms of STOI and PESQ.

show abstract

Section: Experimental Results Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Prathipati,

Chakravarthy

2024

Eng. Res. Express

View full text Add to dashboard Cite

show abstract

“…Various models based on U-Net have been studied, including the Nested U-Net [41] and Wave-U-Net [42], and studies on speech enhancement using these modified models have been conducted. Xiang et al applied self-attention to the Nested U-Net structure, which contains skip connections at each stage of the encoder and decoder, to perform speech enhancement using contextual information at various scales [43]. Craig et al used Wave-U-Net, which applies U-Net to a one-dimensional time domain, to separate vocals and accompaniments in music for speech enhancement [44].…”

Section: B Speech Enhancementmentioning

confidence: 99%

SELD U-Net: Joint Optimization of Sound Event Localization and Detection With Noise Reduction

Shin,

Kim,

Choi

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Sound event localization and detection (SELD) is a combined task that classifies acoustic events from audio signals, estimates temporal boundaries, and identifies event locations. With the advancement of industries utilizing audio signals, SELD has been applied in various fields, and deeplearning-based research is being conducted for its effective application. However, current deep-learningbased SELD research focuses mainly on performance improvement in noise-free environments, which leads to performance degradation issues in noisy environments. To address this problem, this study proposes a robust SELD U-Net model that performs SELD in noisy environments. The proposed model combines a U-Net to remove noise and a SELDnet to perform SELD. The proposed model was trained and evaluated using noisy environmental data with various sizes. Consequently, it was confirmed that the proposed model has superior performance compared with existing deep learning-based SELD models in environments with high levels of noise.

show abstract

“…Previous studies have demonstrated that real-valued DNNs offer numerous advantages in acoustic echo cancellation, including efficient acoustic echo cancellation, strong adaptability, high-quality output, excellent real-time performance, and robust scalability. It ensures clear and natural speech signals, making it suitable for a wide range of real-time communication and speech processing applications [29,30]. Building on the works presented in [15], we incorporate the PE module into our model to facilitate the conversion of the complex spectra to a real spectrum.…”

Section: Phase Encoder and Time-frequency Convolution Modulementioning

confidence: 99%

A Multi-Stage Acoustic Echo Cancellation Model Based on Adaptive Filters and Deep Neural Networks

Xu,

He,

Yan

et al. 2023

Electronics

View full text Add to dashboard Cite

The presence of a large amount of echoes significantly impairs the quality and intelligibility of speech during communication. To address this issue, numerous studies and models have been conducted to cancel echo. In this study, we propose a multi-stage acoustic echo cancellation model that utilizes an adaptive filter and a deep neural network. Our model consists of two parts: the Speex algorithm for canceling linear echo, and the multi-scale time-frequency UNet (MSTFUNet) for further echo cancellation. The Speex algorithm takes the far-end reference speech and the near-end microphone signal as inputs, and outputs the signal after linear echo cancellation. MSTFUNet takes the spectra of the far-end reference speech, the near-end microphone signal, and the output of Speex as inputs, and generates the estimated near-end speech spectrum as output. To enhance the performance of the Speex algorithm, we conduct delay estimation and compensation to the far-end reference speech. For MSTFUNet, we employ multi-scale time-frequency processing to extract information from the input spectrum. Additionally, we incorporate an improved time-frequency self-attention to capture time-frequency information. Furthermore, we introduce channel time-frequency attention to alleviate information loss during downsampling and upsampling. In our experiments, we evaluate the performance of our proposed model on both our test set and the blind test set of the Acoustic Echo Cancellation challenge. Our proposed model exhibits superior performance in terms of acoustic echo cancellation and noise reverberation suppression compared to other models.

show abstract

A Nested U-Net With Self-Attention and Dense Connectivity for Monaural Speech Enhancement

Cited by 41 publications

References 43 publications

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

Single channel speech enhancement using time-frequency attention mechanism based nested U-net model

SELD U-Net: Joint Optimization of Sound Event Localization and Detection With Noise Reduction

A Multi-Stage Acoustic Echo Cancellation Model Based on Adaptive Filters and Deep Neural Networks

Contact Info

Product

Resources

About