Investigation of tandem deep belief network approach for phoneme recognition

Zheng, Xin; Wu, Zhiyong; Shen, Binbin; Meng, Helen; Cai, Lianhong

doi:10.1109/icassp.2013.6639138

Cited by 2 publications

(2 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We trained RBMs in the same way as in [12]. For all the DNNs in HMM-DNN phoneme recognition system, a same architecture of 6 hidden layers by 2000 units was applied.…”

Section: Configuration Of Neural Networkmentioning

confidence: 99%

See 1 more Smart Citation

Contrastive auto-encoder for phoneme recognition

Zheng

Meng

et al. 2014

2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Speech data typically contains task irrelevant information lying within features. Specifically, phonetic information, speaker characteristic information, emotional information and noise are always mixed together and tend to impair one another for certain task. We propose a new type of auto-encoder for feature learning called contrastive auto-encoder. Unlike other variants of auto-encoders, contrastive auto-encoder is able to leverage class labels in constructing its representation layer. We achieve this by modeling two autoencoders together and making their differences contribute to the total loss function. The transformation built with contrastive auto-encoder can be seen as a task-specific and invariant feature learner. Our experiments on TIMIT clearly show the superiority of the feature extracted from contrastive auto-encoder over original acoustic feature, feature extracted from deep auto-encoder, and feature extracted from a model that contrastive auto-encoder originates from.

show abstract

“…We trained RBMs in the same way as in [12]. For all the DNNs in HMM-DNN phoneme recognition system, a same architecture of 6 hidden layers by 2000 units was applied.…”

Section: Configuration Of Neural Networkmentioning

confidence: 99%

“…The DNNs were trained with cross-entropy criterion and SGD+momentum. All the other hyper-parameters were the same with [12].…”

Section: Configuration Of Neural Networkmentioning

confidence: 99%