Speaker-Invariant Training Via Adversarial Learning

Meng, Ziyang; Li, Jinyu; Chen, Zhuo; Zhao, Yang; Mazalov, Vadim; Gang, Yifan; Juang, Biing‐Hwang

doi:10.1109/icassp.2018.8461932

Cited by 94 publications

(63 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Different from domain adaptation, speaker adaptation has only access to limited adaptation data from target speakers and has no access to the source-domain data. Many techniques have been proposed for speaker adaptation of deep acoustic models, such as regularization-based [25,26,27], transformation-based [28,29], singular value decomposition-based [30,31], subspace-based [32,33] and adversarial learning-based [34,35] approaches. Among these, KL divergence (KLD) regularization [25] is one of the most popular methods to prevent the adapted model from overfitting the limited speaker data.…”

Section: Conditional T/s Learning For Speaker Adaptationmentioning

confidence: 99%

Conditional Teacher-student Learning

Meng

Zhao

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

The teacher-student (T/S) learning has been shown to be effective for a variety of problems such as domain adaptation and model compression. One shortcoming of the T/S learning is that a teacher model, not always perfect, sporadically produces wrong guidance in form of posterior probabilities that misleads the student model towards a suboptimal performance. To overcome this problem, we propose a conditional T/S learning scheme, in which a "smart" student model selectively chooses to learn from either the teacher model or the ground truth labels conditioned on whether the teacher can correctly predict the ground truth. Unlike a naive linear combination of the two knowledge sources, the conditional learning is exclusively engaged with the teacher model when the teacher model's prediction is correct, and otherwise backs off to the ground truth. Thus, the student model is able to learn effectively from the teacher and even potentially surpass the teacher. We examine the proposed learning scheme on two tasks: domain adaptation on CHiME-3 dataset and speaker adaptation on Microsoft short message dictation dataset. The proposed method achieves 9.8% and 12.8% relative word error rate reductions, respectively, over T/S learning for environment adaptation and speaker-independent model for speaker adaptation.

show abstract

Section: Conditional T/s Learning For Speaker Adaptationmentioning

confidence: 99%

Conditional Teacher-student Learning

Meng

Zhao

et al. 2019

ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…We additionally provide results on the combined WSJ0+CHiME3 dataset. [11] to use far-field speech from the fifth microphone channel for all sets. We adopt the same input-output setting for CHiME3 as WSJ0.…”

Section: Methodsmentioning

confidence: 99%

NIESR: Nuisance Invariant End-to-End Speech Recognition

Hsu

Jaiswal

Natarajan³

2019

Interspeech 2019

View full text Add to dashboard Cite

Deep neural network models for speech recognition have achieved great success recently, but they can learn incorrect associations between the target and nuisance factors of speech (e.g., speaker identities, background noise, etc.), which can lead to overfitting. While several methods have been proposed to tackle this problem, existing methods incorporate additional information about nuisance factors during training to develop invariant models. However, enumeration of all possible nuisance factors in speech data and the collection of their annotations is difficult and expensive. We present a robust training scheme for end-to-end speech recognition that adopts an unsupervised adversarial invariance induction framework to separate out essential factors for speech-recognition from nuisances without using any supplementary labels besides the transcriptions. Experiments show that the speech recognition model trained with the proposed training scheme achieves relative improvements of 5.48% on WSJ0, 6.16% on CHiME3, and 6.61% on TIMIT dataset over the base model. Additionally, the proposed method achieves a relative improvement of 14.44% on the combined WSJ0+CHiME3 dataset.

show abstract

“…The frame-level cluster labels are regarded as pseudo phone labels to support supervised DNN training. Motivated by successful applications of adversarial training [20] in a wide range of domain invariant learning tasks [21][22][23][24], this work proposes to add an auxiliary adversarial speaker classification task to explicitly target speaker-invariant feature learning. After speaker adversarial multi-task learning (AMTL) DNN training, softmax PG representation from pseudo phone classification task is used to infer subword unit sequences.…”

Section: System Description 21 General Frameworkmentioning

confidence: 99%

Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

2019

View full text Add to dashboard Cite

This study addresses the problem of unsupervised subword unit discovery from untranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech 2019, building text-to-speech systems without text labels. In this work, unit discovery is formulated as a pipeline of phonetically discriminative feature learning and unit inference. One major difficulty in robust unsupervised feature learning is dealing with speaker variation. Here the robustness towards speaker variation is achieved by applying adversarial training and FHVAE based disentangled speech representation learning. A comparison of the two approaches as well as their combination is studied in a DNN-bottleneck feature (DNN-BNF) architecture. Experiments are conducted on ZeroSpeech 2019 and 2017. Experimental results on Ze-roSpeech 2017 show that both approaches are effective while the latter is more prominent, and that their combination brings further marginal improvement in across-speaker condition. Results on ZeroSpeech 2019 show that in the ABX discriminability task, our approaches significantly outperform the official baseline, and are competitive to or even outperform the official topline. The proposed unit sequence smoothing algorithm improves synthesis quality, at a cost of slight decrease in ABX discriminability.

show abstract

Speaker-Invariant Training Via Adversarial Learning

Cited by 94 publications

References 26 publications

Conditional Teacher-student Learning

Conditional Teacher-student Learning

NIESR: Nuisance Invariant End-to-End Speech Recognition

Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

Contact Info

Product

Resources

About