Unsupervised acoustic and language model training with small amounts of labelled data

Novotney, Scott; Schwartz, Richard; Ma, Junxia

doi:10.1109/icassp.2009.4960579

Cited by 52 publications

(36 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…For more than a decade, the term unsupervised training in acoustic modeling has referred to using lightly supervised models to generate noisy transcriptions for unannotated speech, which are fed back for subsequent retraining [11,12]. In the past five years, however, truly unsupervised subword acoustic model training has been attempted using various bottom-up strategies, including Gaussian mixture-based universal background models [7], successive state splitting algorithms for hidden Markov models (HMM) [13], traditional estimation of subword HMMs [14], discriminative clustering objectives [15], and non-parametric Bayesian estimation of HMMs [16].…”

Section: Related Workmentioning

confidence: 99%

Weak top-down constraints for unsupervised acoustic model training

Jansen

Thomas

Heřmanský

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

Typical supervised acoustic model training relies on strong top-down constraints provided by dynamic programming alignment of the input observations to phonetic sequences derived from orthographic word transcripts and pronunciation dictionaries. This paper investigates a much weaker form of top-down supervision for use in place of transcripts and dictionaries in the zero resource setting. Our proposed constraints, which can be produced using recent spoken term discovery systems, come in the form of pairs of isolated word examples that share the same unknown type. For each pair, we perform a dynamic programming alignment of the acoustic observations of the two constituent examples, generating an inventory of cross-speaker frame pairs that each provide evidence that the same subword unit model should account for them. We find these weak top-down constraints are capable of improving model speaker independence by up to 57% relative over bottom-up training alone.

show abstract

Section: Related Workmentioning

confidence: 99%

Weak top-down constraints for unsupervised acoustic model training

Jansen

Thomas

Heřmanský

2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

View full text Add to dashboard Cite

show abstract

“…One typical set of examples are the self-training methods [12,13,14,15]. In this method the transcribed data is used to construct a seed model, with which to decode the untranscribed data at the second stage.…”

Section: Introductionmentioning

confidence: 99%

Semi-supervised DNN training in meeting recognition

Zhang¹,

Liu

Hain

2014

2014 IEEE Spoken Language Technology Workshop (SLT)

View full text Add to dashboard Cite

Training acoustic models for ASR requires large amounts of labelled data which is costly to obtain. Hence it is desirable to make use of unlabelled data. While unsupervised training can give gains for standard HMM training, it is more difficult to make use of unlabelled data for discriminative models. This paper explores semi-supervised training of Deep Neural Networks (DNN) in a meeting recognition task. We first analyse the impact of imperfect transcription on the DNN and the ASR performance. As labelling error is the source of the problem, we investigate two options available to reduce that: selecting data with fewer errors, and changing the dependence on noise by reducing label precision. Both confidence based data selection and label resolution change are explored in the context of two scenarios of matched and unmatched unlabelled data. We introduce improved DNN based confidence score estimators and show their performance on data selection for both scenarios. Confidence score based data selection was found to yield up to 14.6% relative WER reduction, while better balance between label resolution and recognition hypothesis accuracy allowed further WER reductions by 16.6% relative in the mismatched scenario.

show abstract

“…Initial Unlabeled Data These self-training methods have been studied in GMM-based acoustic models [76,77,48,146,105,153]. In recent studies [136,45,60,82], self-training methods are also used in DNN-based acoustic model training.…”

Section: Initial Labeled Datamentioning

confidence: 99%

“…Following the conventional self training [76,77,48,146,105,153] approach, we first train an initial DNN-HMM system using the training data, and decode on the test data. For data selection, we pick utterances with the highest average per-frame decoding likelihood and add them to the training data.…”

Section: Comparison To Self Trainingmentioning

confidence: 99%

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Liu

Kirchhoff

2016

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Graph-based semi-supervised learning (SSL) is a widely used semi-supervised learning method in which the labeled data and unlabeled data are jointly represented as a weighted graph, and the information is propagated from the labeled data to the unlabeled data. The key assumption that graph-based SSL makes is that data samples lie on a low dimensional manifold, where samples that are close to each other are expected to have the same class label. More importantly, by exploiting the relationship between training and test samples, graph-based SSL implicitly adapts to the test data.In this thesis, we address several key challenges in applying graph-based SSL to acoustic modeling. We first investigate and compare several state-of-the-art graph-based SSL algorithms on a benchmark dataset. In addition, we propose novel graph construction methods that allow graph-based SSL to handle variable-length input features. We next investigate the efficacy of graph-based SSL in context of a fully-fledged DNN-based ASR system. We compare two different integration frameworks for graph-based learning. First, we propose a lattice-based late integration framework that combines graph-based SSL with the DNN-based acoustic modeling and evaluate the framework on continuous word recognition tasks. Second, we propose an early integration framework using neural graph embeddings and compare two different neural graph embedding features that capture the information of the manifold at different levels. The embedding features are used as input to a DNN system and are shown to outperform the conventional acoustic feature inputs on several medium-to-large vocabulary conversational speech recognition tasks.

show abstract

Unsupervised acoustic and language model training with small amounts of labelled data

Cited by 52 publications

References 6 publications

Weak top-down constraints for unsupervised acoustic model training

Weak top-down constraints for unsupervised acoustic model training

Semi-supervised DNN training in meeting recognition

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Contact Info

Product

Resources

About