Cycle-Consistent Speech Enhancement

Meng, Ziyang; Li, Jinyu; Gong, Yifan; Juang, Biing‐Hwang

doi:10.21437/interspeech.2018-2409

Cited by 46 publications

(28 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…ASR suffers from performance degradation when a well-trained acoustic model is applied in a new domain [19]. T/S learning [3,8,9] and adversarial learning [20,21,22,23,24] are two effective approaches that can suppress this domain mismatch by adapting a source-domain acoustic model to target-domain speech. T/S learning is more suited for the situation where unlabeled parallel data is available for adaptation, 2 in which a sequence of source-domain speech features is fed as the input to a source-domain teacher model and a parallel sequence of target-domain features is at the input to the target-domain student model to optimize the student model parameters by minimizing the T/S loss in Eq.…”

Section: Conditional T/s Learning For Domain Adaptationmentioning

confidence: 99%

Conditional Teacher-student Learning

Meng

Zhao

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

The teacher-student (T/S) learning has been shown to be effective for a variety of problems such as domain adaptation and model compression. One shortcoming of the T/S learning is that a teacher model, not always perfect, sporadically produces wrong guidance in form of posterior probabilities that misleads the student model towards a suboptimal performance. To overcome this problem, we propose a conditional T/S learning scheme, in which a "smart" student model selectively chooses to learn from either the teacher model or the ground truth labels conditioned on whether the teacher can correctly predict the ground truth. Unlike a naive linear combination of the two knowledge sources, the conditional learning is exclusively engaged with the teacher model when the teacher model's prediction is correct, and otherwise backs off to the ground truth. Thus, the student model is able to learn effectively from the teacher and even potentially surpass the teacher. We examine the proposed learning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker adaptation on Microsoft short message dictation dataset. The proposed method achieves 9.8% and 12.8% relative word error rate reductions, respectively, over T/S learning for environment adaptation and speaker-independent model for speaker adaptation.

show abstract

Section: Conditional T/s Learning For Domain Adaptationmentioning

confidence: 99%

Conditional Teacher-student Learning

Meng

Zhao

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Identity-mapping loss: We regularize Generators G and F to be close to identity mappings by minimizing identity-mapping loss as in [21]. This loss preserves the compositions ((i.e., linguistic information) of the input source features and the target ones [22], [23], and helps the generators better map the target distribution.…”

Section: Cycle-consistent Relativistic Gan For Sementioning

confidence: 99%

Improved Relativistic Cycle-Consistent GAN With Dilated Residual Network and Multi-Attention for Speech Enhancement

Wang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Generative adversarial networks (GANs) have been increasingly used as feature mapping functions in speech enhancement, in which the noisy speech features are transformed to the clean ones through the generators. This paper proposes a novel speech enhancement model based on a cycle-consistent relativistic GAN with Dilated Residual Networks and a Multi-attention mechanism. Using the adversarial loss, improved cycle-consistency losses, and an identity-mapping loss, a noisy-to-clean generator G and an inverse clean-to-noisy generator F simultaneously learn the forward and backward mappings between the source and target domains. To guarantee the stability of the training process, we replace vanilla GAN loss with relativistic average GAN loss and use spectral normalization in discriminators so that they conform to Lipschitz continuity. Furthermore, we employ two attention-based components as multi-attention mechanism to reduce importing signal distortion: attention U-net gates and dilated residual self-attention blocks. By employing these components, our proposed generators can capture long-term inner dependencies between elements of speech features and further preserve linguistic information. Experimental results on a public dataset indicate that the proposed model achieves state-of-the-art speech enhancement performance, especially in reducing speech distortion and improving signal overall quality. Compared with the representative GAN-based approaches, the proposed method significantly achieves the best performance in terms of STOI, CSIG, COVL, and CBAK objective metrics. Moreover, we demonstrate the contribution of each proposed component including relativistic average loss, attention U-net gate, self-attention layers, spectral normalization, and dilation operation by ten comparison systems. INDEX TERMS speech enhancement, cycle-consistent GAN, relativistic average loss, multi-attention, dilated residual network, U-net.

show abstract

“…Instead of explicitly minimizing the L1\L2 losses, which can cause over-smoothing results, the discriminator provides a high-level abstract measurement of the "realness" of the generated images. This idea has been used in SE tasks with parallel [18][19][20][21][22][23] or nonparallel corpora [24,25].…”

Section: Related Workmentioning

confidence: 99%

Noise Adaptive Speech Enhancement Using Domain Adversarial Training

et al. 2019

View full text Add to dashboard Cite

In this study, we propose a novel noise adaptive speech enhancement (SE) system, which employs a domain adversarial training (DAT) approach to tackle the issue of a noise type mismatch between the training and testing conditions. Such a mismatch is a critical problem in deep-learning-based SE systems. A large mismatch may cause a serious performance degradation to the SE performance. Because we generally use a well-trained SE system to handle various unseen noise types, a noise type mismatch commonly occurs in real-world scenarios. The proposed noise adaptive SE system contains an encoder-decoderbased enhancement model and a domain discriminator model. During adaptation, the DAT approach encourages the encoder to produce noise-invariant features based on the information from the discriminator model and consequentially increases the robustness of the enhancement model to unseen noise types. Herein, we regard stationary noises as the source domain (with the ground truth of clean speech) and non-stationary noises as the target domain (without the ground truth). We evaluated the proposed system on TIMIT sentences. The experiment results show that the proposed noise adaptive SE system successfully provides significant improvements in PESQ (19.0%), SSNR (39.3%), and STOI (27.0%) over the SE system without an adaptation.

show abstract

Cycle-Consistent Speech Enhancement

Cited by 46 publications

References 40 publications

Conditional Teacher-student Learning

Conditional Teacher-student Learning

Improved Relativistic Cycle-Consistent GAN With Dilated Residual Network and Multi-Attention for Speech Enhancement

Noise Adaptive Speech Enhancement Using Domain Adversarial Training

Contact Info

Product

Resources

About