Attention Wave-U-Net for Speech Enhancement

Giri, Ritwik; Isik, Umut; Krishnaswamy, Arvindh

doi:10.1109/waspaa.2019.8937186

Cited by 90 publications

(43 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Interpolation + convolution was proposed as an alternative to avoid transposed convolution artifacts [15]. It has been used, e.g., for music source separation [2], speech enhancement [22] and neural vocoding [14]. While interpolation (e.g., linear or nearest neighbor) is effectively upsampling the signal, the subsequent convolution further transforms the upsampled signal with learnable weights.…”

Section: Intepolation Upsamplersmentioning

confidence: 99%

Upsampling Artifacts in Neural Audio Synthesis

Pons

Pascual

Cengarle

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

A number of recent advances in neural audio synthesis rely on upsampling layers, which can introduce undesired artifacts. In computer vision, upsampling artifacts have been studied and are known as checkerboard artifacts (due to their characteristic visual pattern). However, their effect has been overlooked so far in audio processing. Here, we address this gap by studying this problem from the audio signal processing perspective. We first show that the main sources of upsampling artifacts are: (i) the tonal and filtering artifacts introduced by problematic upsampling operators, and (ii) the spectral replicas that emerge while upsampling. We then compare different upsampling layers, showing that nearest neighbor upsamplers can be an alternative to the problematic (but state-of-the-art) transposed and subpixel convolutions which are prone to introduce tonal artifacts.

show abstract

Section: Intepolation Upsamplersmentioning

confidence: 99%

Upsampling Artifacts in Neural Audio Synthesis

Pons

Pascual

Cengarle

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…In the joint training process, an attention mechanism is added between the input in the SED module and the output passing through each convolutional layer and maxpooling layer. We exploit an attention mechanism of a similar form used in [39,40], but with different stride sizes of each convolutional layer, normalization, and skip-connection. As shown in Figure 3, i is set as a variable representing the order of blocks (convolutional and max-pooling layers) to which the attention module is added.…”

Section: Attention Mechanismmentioning

confidence: 99%

Attention-Based Joint Training of Noise Suppression and Sound Event Detection for Noise-Robust Classification

Son

Chang

2021

Sensors

View full text Add to dashboard Cite

Sound event detection (SED) recognizes the corresponding sound event of an incoming signal and estimates its temporal boundary. Although SED has been recently developed and used in various fields, achieving noise-robust SED in a real environment is typically challenging owing to the performance degradation due to ambient noise. In this paper, we propose combining a pretrained time-domain speech-separation-based noise suppression network (NS) and a pretrained classification network to improve the SED performance in real noisy environments. We use group communication with a context codec method (GC3)-equipped temporal convolutional network (TCN) for the noise suppression model and a convolutional recurrent neural network for the SED model. The former significantly reduce the model complexity while maintaining the same TCN module and performance as a fully convolutional time-domain audio separation network (Conv-TasNet). We also do not update the weights of some layers (i.e., freeze) in the joint fine-tuning process and add an attention module in the SED model to further improve the performance and prevent overfitting. We evaluate our proposed method using both simulation and real recorded datasets. The experimental results show that our method improves the classification performance in a noisy environment under various signal-to-noise-ratio conditions.

show abstract

“…In addition, the far-end information, which helps to subtract an acoustic echo, is extracted by passing through the auxiliary encoder, and it is transmitted as compressed latent features of the mixtures through an element-wise multiplication. To build a more effective connection, an attention mechanism [ 25 ] is applied to enable an efficient information delivery, as described in detail in Figure 3 . The latent features of the far-end and compressed multi-channel mixture are mapped to the intermediate feature space.…”

Section: Proposed Multi-channel Cross-tower With Attention Mechanimentioning

confidence: 99%

Multi-TALK: Multi-Microphone Cross-Tower Network for Jointly Suppressing Acoustic Echo and Background Noise

Park

Chang

2020

Sensors

View full text Add to dashboard Cite

In this paper, we propose a multi-channel cross-tower with attention mechanisms in latent domain network (Multi-TALK) that suppresses both the acoustic echo and background noise. The proposed approach consists of the cross-tower network, a parallel encoder with an auxiliary encoder, and a decoder. For the multi-channel processing, a parallel encoder is used to extract latent features of each microphone, and the latent features including the spatial information are compressed by a 1D convolution operation. In addition, the latent features of the far-end are extracted by the auxiliary encoder, and they are effectively provided to the cross-tower network by using the attention mechanism. The cross tower network iteratively estimates the latent features of acoustic echo and background noise in each tower. To improve the performance at each iteration, the outputs of each tower are transmitted as the input for the next iteration of the neighboring tower. Before passing through the decoder, to estimate the near-end speech, attention mechanisms are further applied to remove the estimated acoustic echo and background noise from the compressed mixture to prevent speech distortion by over-suppression. Compared to the conventional algorithms, the proposed algorithm effectively suppresses the acoustic echo and background noise and significantly lowers the speech distortion.

show abstract

Attention Wave-U-Net for Speech Enhancement

Cited by 90 publications

References 17 publications

Upsampling Artifacts in Neural Audio Synthesis

Upsampling Artifacts in Neural Audio Synthesis

Attention-Based Joint Training of Noise Suppression and Sound Event Detection for Noise-Robust Classification

Multi-TALK: Multi-Microphone Cross-Tower Network for Jointly Suppressing Acoustic Echo and Background Noise

Contact Info

Product

Resources

About