Using Recurrences in Time and Frequency within U-net Architecture for Speech Enhancement

Grzywalski, Tomasz; Drgas, Szymon

doi:10.1109/icassp.2019.8682830

Cited by 15 publications

(9 citation statements)

References 19 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The FGRU and TGRU used in TRU-Net is similar to the work in [32]. They used bidirectional long short-term memory (bi-LSTM) networks on the frequency-axis and the time-axis combined with 2D-CNN-based U-Net.…”

Section: Relation To Prior Workmentioning

confidence: 99%

“…They used bidirectional long short-term memory (bi-LSTM) networks on the frequency-axis and the time-axis combined with 2D-CNN-based U-Net. The difference is that bi-LSTM was utilized to increase performance in [32], whereas we employ FGRU and uni-directional TGRU to better handle the online inference scenario combined with the proposed lightweight 1D-CNN-based (frequency-axis) U-Net.…”

Section: Relation To Prior Workmentioning

confidence: 99%

See 1 more Smart Citation

Real-Time Denoising and Dereverberation wtih Tiny Recurrent U-Net

Choi

Park

Lee³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Modern deep learning-based models have seen outstanding performance improvement with speech enhancement tasks. The number of parameters of state-of-the-art models, however, is often too large to be deployed on devices for real-world applications. To this end, we propose Tiny Recurrent U-Net (TRU-Net), a lightweight online inference model that matches the performance of current state-ofthe-art models. The size of the quantized version of TRU-Net is 362 kilobytes, which is small enough to be deployed on edge devices. In addition, we combine the small-sized model with a new masking method called phase-aware β-sigmoid mask, which enables simultaneous denoising and dereverberation. Results of both objective and subjective evaluations have shown that our model can achieve competitive performance with the current state-of-the-art models on benchmark datasets using fewer parameters by orders of magnitude.

show abstract

Section: Relation To Prior Workmentioning

confidence: 99%

Section: Relation To Prior Workmentioning

confidence: 99%

Real-Time Denoising and Dereverberation wtih Tiny Recurrent U-Net

Choi

Park

Lee³

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Considering that many state-of-the-art CNN speech enhancement methods have shortcuts [24], [25], we also investigate the effect of the proposed SE mechanisms when combined with shortcut-based CNNs. We build a shortcutbased convolutional network (SCN) that adds shortcuts between the corresponding layers in the encoder and decoder of the CNN used before and then evaluate its performance with and without SE mechanisms.…”

Section: Se Generalizationmentioning

confidence: 99%

“…Tan and Wang [24] proposed the convolutional recurrent network (CRN), which inserted two long short-term memory (LSTM) layers between the encoder and the decoder of the FCN. Grzywalski and Drgas [25] added gated recurrent unit (GRU) layers into each building block of the FCN. These models improve the representational capability by exploiting the temporal modeling capability of RNN.…”

Section: Introductionmentioning

confidence: 99%

“…Motivated by these works, we introduce SE as the attention mechanism, and combine it with RCED, thus solving the problem that RCED has difficulty in effectively exploiting global information [25] or explicitly judging the importance of different features. Considering that the information in time-frequency points are also of great importance, we propose a spatial SE (SSE) mechanism that provides weights spatially.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Combining Multi-Perspective Attention Mechanism With Convolutional Networks for Monaural Speech Enhancement

Lan

Lyu

et al. 2020

IEEE Access

View full text Add to dashboard Cite

The redundant convolutional encoder-decoder network has been proven useful in speech enhancement tasks. This network can capture the localized time-frequency details of speech signals through the fully convolutional network structure and the feature selection capability that results from the encoderdecoder mechanism. However, extracting informative features, which we regard as important for the representational capability of speech enhancement models, is not considered explicitly. To solve this problem, we introduce the attention mechanism into the convolutional encoder-decoder model to explicitly emphasize useful information from three aspects, namely, channel, space, and concurrent space-and-channel. Furthermore, the attention operation is specifically achieved through the squeeze-and-excitation mechanism and its variants. The model can adaptively emphasize valuable information and suppress useless ones by assigning weights from different perspectives according to global information, thereby improving its representational capability. Experimental results show that the proposed attention mechanisms can employ a small fraction of parameters to effectively improve the performance of CNN-based models compared with their normal versions, and generalize well to unseen noises, signal-to-noise ratios (SNR) and speakers. Among these mechanisms, the concurrent space-channel-wise attention exhibits the most significant improvement. And when comparing with the state-of-the-art, they can produce comparable or better results. We also integrate the proposed attention mechanisms with other convolutional neural network (CNN)-based models and gain performance. Moreover, we visualize the enhancement results to show the effect of the attention mechanisms more clearly.

show abstract