Multilingual MLP features for low-resource LVCSR systems

Thomas, Samuel; Ganapathy, Sriram; Heřmanský, Hynek

doi:10.1109/icassp.2012.6288862

Cited by 121 publications

(124 citation statements)

References 8 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…In previous work, we proposed the use of a modified neural network in which the final phoneme-dependent soft-max layer is replaced [11]. We use this technique in this work for adapting the DNN as well.…”

Section: Updating the Dnn With Monolingual Datamentioning

confidence: 99%

“…2.1. Training is performed using a combined phoneme set size of 52 derived from a count-based mapping [8].…”

Section: Dnn Pre-training With Multilingual Datamentioning

confidence: 99%

“…While this work is related to several recent approaches [8,9,10,11,12], we use two different techniques to derive better features and improve acoustic model training in low resource settings: data driven features extracted using deep neural networks (DNN) This work was supported in parts by the DARPA RATS project D10PC0015, IARPA BABEL project W911NF12-C-0013, and by the Johns Hopkins Center of Excellence in Human Language Technologies. and semi-supervised training.…”

Section: Introductionmentioning

confidence: 99%

“…multilingual data or data from the same language but collected under different settings [6,7]. Features corresponding to limited task specific data are then derived using the trained MLP for ASR [8,9,10,11,12]. We build on this front-end-based approach since features produced using these front-ends can further improve performance in low-resource settings when combined with other ASR modeling techniques.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Deep neural network features and semi-supervised training for low resource speech recognition

Thomas

Seltzer

Church³

et al. 2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

Self Cite

138

100

View full text Add to dashboard Cite

We propose a new technique for training deep neural networks (DNNs) as data-driven feature front-ends for large vocabulary continuous speech recognition (LVCSR) in low resource settings. To circumvent the lack of sufficient training data for acoustic modeling in these scenarios, we use transcribed multilingual data and semi-supervised training to build the proposed feature front-ends. In our experiments, the proposed features provide an absolute improvement of 16% in a low-resource LVCSR setting with only one hour of in-domain training data. While close to three-fourths of these gains come from DNN-based features, the remaining are from semi-supervised training.

show abstract

Section: Updating the Dnn With Monolingual Datamentioning

confidence: 99%

“…2.1. Training is performed using a combined phoneme set size of 52 derived from a count-based mapping [8].…”

Section: Dnn Pre-training With Multilingual Datamentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Deep neural network features and semi-supervised training for low resource speech recognition

Thomas

Seltzer

Church³

et al. 2013

2013 IEEE International Conference on Acoustics, Speech and Signal Processing

Self Cite

138

100

View full text Add to dashboard Cite

show abstract

“…These posteriors are then used as discriminative features for the conventional HMM/GMM (Hidden Markov Model / Gaussian Mixture Model) models of the target language. To improve performance, the source MLPs can be adapted to fit the target language better [5].…”

Section: Introductionmentioning

confidence: 99%

Cross-Lingual Phone Mapping for Large Vocabulary Speech Recognition of Under-Resourced Languages

Hai

Xiao

Chng

et al. 2014

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

SUMMARYThis paper presents a novel acoustic modeling technique of large vocabulary automatic speech recognition for under-resourced languages by leveraging well-trained acoustic models of other languages (called source languages). The idea is to use source language acoustic model to score the acoustic features of the target language, and then map these scores to the posteriors of the target phones using a classifier. The target phone posteriors are then used for decoding in the usual way of hybrid acoustic modeling. The motivation of such a strategy is that human languages usually share similar phone sets and hence it may be easier to predict the target phone posteriors from the scores generated by source language acoustic models than to train from scratch an under-resourced language acoustic model. The proposed method is evaluated using on the Aurora-4 task with less than 1 hour of training data. Two types of source language acoustic models are considered, i.e. hybrid HMM/MLP and conventional HMM/GMM models. In addition, we also use triphone tied states in the mapping. Our experimental results show that by leveraging well trained Malay and Hungarian acoustic models, we achieved 9.0% word error rate (WER) given 55 minutes of English training data. This is close to the WER of 7.9% obtained by using the full 15 hours of training data and much better than the WER of 14.4% obtained by conventional acoustic modeling techniques with the same 55 minutes of training data.

show abstract

Automatic Speech Recognition

Renals¹,

King²

2010

The Handbook of Phonetic Sciences

View full text Add to dashboard Cite

In most of state-of-the-art speech recognition systems, Gaussian mixture models (GMMs) are used to model the density of the emitting states in the hidden Markov models (HMMs). In a conventional system, the model parameters of each GMM are estimated directly and independently given the alignment. This results a large number of model parameters to be estimated, and consequently, a large amount of training data is required to fit the model. In addition, different sources of acoustic variability that impact the accuracy of a recogniser such as pronunciation variation, accent, speaker factor and environmental noise are only weakly modelled and factorized by adaptation techniques such as maximum likelihood linear regression (MLLR), maximum a posteriori adaptation (MAP) and vocal tract length normalisation (VTLN). In this thesis, we will discuss an alternative acoustic modelling approach -the subspace Gaussian mixture model (SGMM), which is expected to deal with these two issues better. In an SGMM, the model parameters are derived from low-dimensional model and speaker subspaces that can capture phonetic and speaker correlations. Given these subspaces, only a small number of state-dependent parameters are required to derive the corresponding GMMs. Hence, the total number of model parameters can be reduced, which allows acoustic modelling with a limited amount of training data. In addition, the SGMM-based acoustic model factorizes the phonetic and speaker factors and within this framework, other source of acoustic variability may also be explored.In this thesis, we propose a regularised model estimation for SGMMs, which avoids overtraining in case that the training data is sparse. We will also take advantage of the structure of SGMMs to explore cross-lingual acoustic modelling for low-resource speech recognition. Here, the model subspace is estimated from out-domain data and ported to the target language system. In this case, only the state-dependent parameters need to be estimated which relaxes the requirement of the amount of training data. To improve the robustness of SGMMs against environmental noise, we propose to apply the joint uncertainty decoding (JUD) technique that is shown to be efficient and effective. We will report experimental results on the Wall Street Journal (WSJ) database and GlobalPhone corpora to evaluate the regularisation and cross-lingual modelling of SGMMs. Noise compensation using JUD for SGMM acoustic models is evaluated on the Aurora 4 database.iii AcknowledgementsFirst and foremost, I would like to thank my supervisor Prof. Steve Renals for his expert guidance, deep insight in the speech technology, patience and encouragement all the way through my doctoral study. It is a great pleasure to work with this wonderful mentor. I also owe a huge gratitude to Dr. Arnab Ghoshal, a colleague, friend and advisor, without whom, I would definitely have a much tougher experience to reach this thesis. Thanks also to KK Chin -my industry supervisor in Toshiba Cambridge research laboratory (CRL). It is a...

show abstract

Multilingual MLP features for low-resource LVCSR systems

Cited by 121 publications

References 8 publications

Deep neural network features and semi-supervised training for low resource speech recognition

Deep neural network features and semi-supervised training for low resource speech recognition

Cross-Lingual Phone Mapping for Large Vocabulary Speech Recognition of Under-Resourced Languages

Automatic Speech Recognition

Contact Info

Product

Resources

About