Multichannel Speech Enhancement by Raw Waveform-mapping using Fully Convolutional Networks

Liu, Chang-Le; Fu, Szu-Wei; Li, You-Jin; Huang, Jen-Wei; Wang, Hsin‐Min; Tsao, Yu

doi:10.48550/arxiv.1909.11909

Cited by 3 publications

(2 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although there are time-domain approaches that use multi-microphone modeling for speech enhancement and source separation [26]- [28], their effectiveness in environments with moderate to strong reverberation is not yet established [29]. In addition, our study tightly integrates multi-microphone complex spectral mapping with beamforming and post-filtering.…”

Section: Introductionmentioning

confidence: 99%

Multi-Microphone Complex Spectral Mapping for Speech Dereverberation

Wang

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

This study proposes a multi-microphone complex spectral mapping approach for speech dereverberation on a fixed array geometry. In the proposed approach, a deep neural network (DNN) is trained to predict the real and imaginary (RI) components of direct sound from the stacked reverberant (and noisy) RI components of multiple microphones. We also investigate the integration of multi-microphone complex spectral mapping with beamforming and post-filtering. Experimental results on multi-channel speech dereverberation demonstrate the effectiveness of the proposed approach.

show abstract

Section: Introductionmentioning

confidence: 99%

Multi-Microphone Complex Spectral Mapping for Speech Dereverberation

Wang

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…We note that such mismatch also exists in many other speech processing tasks, such as speech separation [26], where we observe that, by incorporating time-domain loss function [27], one can improve the output speech quality. More recently, deep learning approach to speech enhancement methods with time-domain raw waveform outputs [28,29] have also been investigated. However, we note that time-domain loss function has not been well explored in speech synthesis, which will be the focus of this paper.…”

Section: Introductionmentioning

confidence: 99%

WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss

Li¹,

Şişman²,

Bao³

et al. 2020

Preprint

View full text Add to dashboard Cite

Tacotron-based text-to-speech (TTS) systems directly synthesize speech from text input. Such frameworks typically consist of a feature prediction network that maps character sequences to frequency-domain acoustic features, followed by a waveform reconstruction algorithm or a neural vocoder that generates the time-domain waveform from acoustic features. As the loss function is usually calculated only for frequency-domain acoustic features, that doesn't directly control the quality of the generated time-domain waveform. To address this problem, we propose a new training scheme for Tacotron-based TTS, referred to as WaveTTS, that has 2 loss functions: 1) timedomain loss, denoted as the waveform loss, that measures the distortion between the natural and generated waveform; and 2) frequency-domain loss, that measures the Mel-scale acoustic feature loss between the natural and generated acoustic features. WaveTTS ensures both the quality of the acoustic features and the resulting speech waveform. To our best knowledge, this is the first implementation of Tacotron with joint time-frequency domain loss. Experimental results show that the proposed framework outperforms the baselines and achieves high-quality synthesized speech.

show abstract

Distributed Microphone Speech Enhancement based on Deep Learning

Wang

Liang²,

Hung

et al. 2019

Preprint

Self Cite

View full text Add to dashboard Cite

Speech-related applications deliver inferior performance in complex noise environments. Therefore, this study primarily addresses this problem by introducing speech-enhancement (SE) systems based on deep neural networks (DNNs) applied to a distributed microphone architecture. The first system constructs a DNN model for each microphone to enhance the recorded noisy speech signal, and the second system combines all the noisy recordings into a large feature structure that is then enhanced through a DNN model. As for the third system, a channel-dependent DNN is first used to enhance the corresponding noisy input, and all the channel-wise enhanced outputs are fed into a DNN fusion model to construct a nearly clean signal. All the three DNN SE systems are operated in the acoustic frequency domain of speech signals in a diffuse-noise field environment. Evaluation experiments were conducted on the Taiwan Mandarin Hearing in Noise Test (TMHINT) database, and the results indicate that all the three DNN-based SE systems provide the original noise-corrupted signals with improved speech quality and intelligibility, whereas the third system delivers the highest signal-to-noise ratio (SNR) improvement and optimal speech intelligibility.

show abstract

Multichannel Speech Enhancement by Raw Waveform-mapping using Fully Convolutional Networks

Cited by 3 publications

References 57 publications

Multi-Microphone Complex Spectral Mapping for Speech Dereverberation

Multi-Microphone Complex Spectral Mapping for Speech Dereverberation

WaveTTS: Tacotron-based TTS with Joint Time-Frequency Domain Loss

Distributed Microphone Speech Enhancement based on Deep Learning

Contact Info

Product

Resources

About